Updated LLM prompts and System message
Some checks failed
Push image to registry / build-image (push) Failing after 51s
Some checks failed
Push image to registry / build-image (push) Failing after 51s
This commit is contained in:
@ -10,9 +10,11 @@ This method comes from the idea that the best way to prompt engineer is to ask t
|
||||
|
||||
# Prompts
|
||||
|
||||
Starting sentence is usually :
|
||||
Starting sentence is usually :
|
||||
|
||||
```
|
||||
I’m using some LLM and I would need a prompt and a system message for every use case I will give you.
|
||||
I’m using structured JSON output provided by the openAI API. The output structure is a simple {"type": "object", "properties": {"answer": {"type": "string"}}, "required": ["answer"]}, so only “answer” can be filled. For the input, everything will be given in the prompt. Give me the system message and prompt separately, preferably in text format.
|
||||
```
|
||||
|
||||
## Instagram
|
||||
@ -20,24 +22,41 @@ I’m using some LLM and I would need a prompt and a system message for every us
|
||||
### Instagram Reel caption generation
|
||||
|
||||
```
|
||||
I’m using some LLM and I would need a prompt, a system message and an output format for every use case I will give you.
|
||||
I’m using some LLM and I would need a prompt, a system message for every use case I will give you.
|
||||
The first one is when I’m trying to generate a caption for an instagram Reel. For the moment, I can give the LLM the original instagram reel caption that was downloaded from, and a description by an LLM of the video, or the joke behind it.
|
||||
The caption must be short and well placed with the reel. For example, if the reel is funny, the caption must be short and funny, while still relating to the reel. The caption must not be describint the video like the LLM description does
|
||||
The caption must be short and well placed with the reel. For example, if the reel is funny, the caption must be short and funny, while still relating to the reel. The caption must not be describing the video like the LLM description does (for example this bad example describe the content of the video instead of doing a caption based on the description given : “Three animated friends chilling in the woods at night until someone's phone inevitably starts ringing somewhere nearby... 😅🌲✨” or this one : “This reel shows me trying to make my sad texts shorter with ChatGPT, but it just frustrates me more! 😅😂”).
|
||||
It also shouldn’t begin with something like ‘This reel…’. For example this is a bad output : “This reel hilariously mocks every awkward fan reaction to those intense DCU movie scenes. 🎭 #DCFanDrama”
|
||||
The LLM can add some appropriate hashtags if it wants to and seem appropriate.
|
||||
Sometimes, the original caption will credit the original author, most of the times on twitter like (“credit : t/twitteruser”). Those credit can appear in the generated caption too, But I don’t want any instagram account mention (“@instagramUser”) because most of the time it’s to incite to subscribe to the downloaded reel account. The use of emoji is encouraged, but not too much and it has to not look stupid or too.
|
||||
Sometimes, the original caption will credit the original author, most of the times on twitter like (“credit : t/twitteruser”). Those credit can appear in the generated caption too, But I don’t want any instagram account mention (“@instagramUser”) because usually it’s to incite to subscribe to the downloaded reel account (like “Seen me already ? follow me @instagramUser”). I don’t want long credits too, juste a simple “credit tt/twitteraccount” is enough. Not like this bad example : “Credited via the brilliant mind at tt/batinterface!…”
|
||||
The use of emoji is encouraged, but not too much and it has to not look stupid or too.
|
||||
|
||||
When using it, I encoutered some problems like this one :
|
||||
“Credit to: [Original Creator] for this hilarious video game scene where the characters look suspiciously like Kermit the Frog! 😂”. The [Original creator] is not filled in, I don’t even know if the original caption had one.
|
||||
|
||||
Some caption are just lame and feels like a facebook post. The intended audience here is young.
|
||||
```
|
||||
|
||||
## Video Descriptor
|
||||
|
||||
I’m using some LLM and I would need a prompt and a system message for every use case I will give you.
|
||||
|
||||
The LLM here will be used to describe an Instagram Reel (video). Each screenshot of that video will be described using an LLM, prompt, system message and output format. The description of all the screenshots will be given to this LLM that will try to recreate the video based on the description of the screenshots, and describe the video.
|
||||
The required prompt here is for the LLM that will compile the description into one and try to understand the video and describe it. I’m particularly interested in the joke behind the reel if there is one.
|
||||
|
||||
This is an example of a screenshot description by an LLM : “The image shows a close-up of a person's hands holding what appears to be a brown object with a plastic covering, possibly food wrapped in paper or foil. There is also a small portion visible at the top right corner, which seems to be a red and white label. The focus of the image is on the hands holding the object.”
|
||||
|
||||
Most of the description won’t make sense, so some details should be omitted. For example, one screenshot description could say the main subject is a car, and another one 3 seconds later in the video could say the main subject is a cat. You could say the car transformed into a cat, but it would be safer to assume that one of the description is wrong and the main characted was a cat all along the video because another description in the video also says the main subject is a cat.
|
||||
It is safe to say that most analysed videos will be of bad quality. which means the screenshots description can vary a lot
|
||||
```
|
||||
I’m using some LLM and I would need a prompt and a system message for every use case I will give you.
|
||||
I’m using structured JSON output provided by the openAI API. The output structure is a simple {"type": "object", "properties": {"answer": {"type": "string"}}, "required": ["answer"]}, so only “answer” can be filled. For the input, everything will be given in the prompt. Give me the system message and prompt separately, preferably in text format.
|
||||
The LLM here will be used to describe an Instagram Reel (video). Each screenshot of that video will be described using an LLM, prompt, system message and output format. The description of all the screenshots will be given to this LLM that will try to recreate the video based on the description of the screenshots, and describe the video.
|
||||
The required prompt here is for the LLM that will compile the description into one and try to understand the video and describe it. I’m particularly interested in the joke behind the reel if there is one.
|
||||
|
||||
This is an example of a screenshot description by an LLM : “The image shows a close-up of a person's hands holding what appears to be a brown object with a plastic covering, possibly food wrapped in paper or foil. There is also a small portion visible at the top right corner, which seems to be a red and white label. The focus of the image is on the hands holding the object.”
|
||||
|
||||
The information I can give in the prompts are the screenshots and for each :
|
||||
|
||||
The screenshot number
|
||||
The timestamp in the video of when the screenshot is taken
|
||||
An OCR result (may contain some weird character, the COR is not filtered or cleansed)
|
||||
The LLM description of the screenshot
|
||||
Most of the description won’t make sense, so some details should be omitted. For example, one screenshot description could say the main subject is a car, and another one 3 seconds later in the video could say the main subject is a cat. You could say the car transformed into a cat, but it would be safer to assume that one of the description is wrong and the main characted was a cat all along the video because another description in the video also says the main subject is a cat.
|
||||
It is safe to say that most analysed videos will be of bad quality. which means the screenshots description can vary a lot.
|
||||
|
||||
Found text by OCR and screenshots descriptions can be retrieved to the final video description if it seems coherent.
|
||||
```
|
||||
|
||||
### Screenshot descriptor
|
||||
|
||||
|
Reference in New Issue
Block a user