Avoiding the Mega Prompt: Task Based Structured Prompting
Complex requirements can lead to monolithic or "mega prompts" in Generative AI. This post outlines an approach to breaking up long prompts into structured task specific ones.
The Advent of the "Mega Prompt"
I'm experienced in Natural Language Processing, but with the recent paradigm shift in Large Language Model (LLM) capabilities, I've found new use cases becoming more and more complex.
Organizations can often attempt to solve these complexities in a number of ways:
- They generate long narrative style prompts.
- They have structured prompts (also long) but they require many edits and often intricate weaving of tasks and guidance.
- They have a mix of numbers 1 and 2.
This has led to prompts that can be up to 400 lines long, becoming what I have termed, the "Mega Prompt."
With that said, in this post I will outline an approach to generating task specific workflows that provide more control over the outcomes and delivering results more aligned with business objectives.
Two Prompting Styles
By now organizations have begun to create functional teams to advance prompt engineering with LLMs. As I mentioned above, often this presents itself into three basic styles.
In the next section we will break those down and discuss a systematic, repeatable style to prompt engineering...
Narrative Style Prompting
Narrative style prompts tell a story, usually as a second- or third-person narrator.
Example:
Task: Make a Peanut Butter and Jelly Sandwich
System Prompt:
You are an expert sandwich maker tasked with creating a peanut butter and jelly (PB&J) sandwich. You've made several of these sandwiches before, but find yourself passing your knowledge on to a new "rookie" sandwich maker that will eventually take over your business. Customers have grown to love your PB&J sandwiches and as you've been in business for 30 years, you want to ensure this "rookie" under your mentorship understands the importance of maintaining the company reputation. With that said, you need to teach them how to make a PB&J sandwich in a step-by-step foolproof way. When the rookie tells you they are ready, you will begin your task.
User Prompt: I'm ready!
Example Analysis:
This prompt is easy to read, paints a clear context for the situation and emphasizes the importance of getting the instructions for making the PB&J sandwich correctly. It also instructs the LLM to generate a "fool proof" set of instructions. Now imagine you were faced with a situation where you were the owner of a "catering" business and had to prepare 5 types of sandwiches, 4 appetizers, 5 main course options, and 4 dessert options. This style of prompting could get very long. Especially if you had certain cooking styles, methods of preparation, food allergies to account for, budget constraints, etc.
As you may imagine, this can pose a lot of issues with troubleshooting. You may have to account for things like:
1. Is the narrative well organized?
2. Does it tell a complete story?
3. Was proper grammar used?
4. When presenting "if then, else this" style guidance, does it make sense in context or could it adversely influence other aspects of the task?
Now let's take a look at a structured approach and analyze it as well...
Structured Style Prompting
Structured Style prompts define what I will call "anchors" to guide the LLM to classify parts of the task it's performing, and then use those classifications as a framework to perform it's task well.
Example:
Task: Make a Peanut Butter and Jelly Sandwich
System Prompt:
1
2
3
4
5
6
7
8
9
10
11
12
<role>You are an expert sandwich maker and the owner of a sandwich making business.</role>
<scenario>You are training a new sandwich maker on how to make a sandwich.</scenario>
<type_of_sandwich>Peanut Butter and Jelly (PB&J)</type_of_sandwich>
<task>Using clear, concise, step by step instructions, tell the new sandwich maker on how to make a PB&J sandwich.</task>
<rules>You must respond in the format provided to you in the <response_format/> XML tags.</rules>
<response_format>
[Step by step instructions go here]
Example of response:
1. Do this first.
2. Next to this.
3. etc.
</response_format>User Prompt: Begin!
Example Analysis:
This prompt should provide a more stable and repeatable outcome. Many large language models desire a "specific style" of prompting. For example Anthropic's Claude family of models prefer structured prompts such as the above. You don't have to, but in my experience, the quality of my results increased with prompts of this "structured style."
The Problem Remains
The structured style demonstrates an improvement in performance and quality with the extra benefit of also helping prompt engineers focus on specific areas of the prompt.
However, as good as this is, it has scaling issues as you look at the catering "what if?" scenario I described above.
For this we need a different approach.
A Different Approach: Task Based Structured Prompting
Let's place ourselves in this scenario and ask if we were the trainer or trainee in this situation, how would we approach the tasks at hand. I personally would want to break down the effort into manageable "training modules," each with clearly outlined steps to follow and guidance for how to do it well. Perhaps extra context, etc.? If I received the narrative or structured prompt with everything all once, I would be overwhelmed and potentially have trouble breaking down the tasks.
I've noticed an affect where the LLMs can struggle with this all at once style, and if they succeed, it can, again, be an intricate effort to troubleshoot the prompt etc.
Breaking it Down
Ok with that out of the way, let's look at how, using prompt engineering, we can break the catering scenario's tasks up to achieve better performance, quality, and operational efficiency.
What do we know
- Expert sandwich maker owns a business and is training someone.
- The company is a catering business.
- The sandwich maker finds themselves make a variety of dishes, broken down as:
- 5 types of sandwiches
- 4 appetizers
- 5 main course options
- 4 dessert options.
- The expert also has to train the next person to perform the tasks and do it well while protecting the business' reputation.
It sounds like we have some "overall" things to do and then some very specific things to do. Here's an approach to break this down.
- Group all overarching guidance and context together.
- Break up the catering needs into specific tasks.
- Structure the response format in a consistent manner.
Note: In the following section I will provide examples, while also providing a glimpse into some more advanced prompt engineering techniques.
Overarching Guidance and Context
Top Level
System Template:
1
2
<!-- This is the system prompt you must follow. -->
<system_prompt>{prompt}</system_prompt>Note: For these prompts I will use a "templating approach." Using the scripting language of your choice, you would replace things like, {prompt} with the final prompt text using string substitution._
Common Main Prompt Elements
Role:
1
2
3
4
<!-- This is your role. -->
<role>
You are an expert sandwich maker with 30 years of experience. You also are the owner of a catering business that has a great reputation in the area. You are also well-versed in teaching, mentorship, and imparting wisdom from your life and work experiences to others.
</role>Scenario and Context:
1
2
3
4
5
6
7
8
<!-- This is the scenario and context you find yourself in while performing your tasks.-->
<scenario_and_context>
- You own a catering business.
- Your catering business has just booked an event.
- For the catering event you have been asked to make 5 types of sandwiches, 4 appetizers, 5 main course options, and 4 dessert options.
- While fulfilling this request, you will also train the person who is taking over your business once you retire.
- The person being trained, loves things to be broken down step by step, with clear instructions that account for all steps in their workflow.
</scenario_and_context>Rubric:
1
2
3
4
5
6
7
8
9
10
<!-- This is the rubric that your tasks will be judged on in a pass or fail manner. If at any time your answers to the rubric fail the standard, revise your work to ensure all rubrics are meeting expectations.-->
<rubric>
1. Did the response contain factual, true statements or was anything made up or hallucinated?
2. Did the response use a friendly, yet professional tone?
3. Were the instructions clear and easy to follow?
4. Did the customer greatly enjoy your food and customer service?
5. Did your subcontractors perform well?
5. Did you trainee walk away equipped and confident in their abilities to not only take over the business, but more importantly complete the current catering request?
6. Do you feel confident that you have passed on knowledge, wisdom, and instructions to your trainee well, and that they will succeed?
</rubric>Specific Task Prompting
Task: Create 4 desserts for the catering event.
Prompt Template:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
<!-- This is your subcontracted task.-->
<task type="dessert making">
<!-- This is your subcontracting role -->
<role>You are an expert Pastry Chef, contracted by the owner of a catering business.</role>
<!-- This is information on your scenario and context for your subcontracted task. -->
<scenario_and_context>
1. You have been subcontracted to create desserts for the catering event and also train a new chef in how to make desserts in a step-by-step manner.
2. You have to respond in the format found within the <response_format/>
XML tags.
3. You have hired subcontractors to prepare the individual dishes for the meal stages requested.
</scenario_and_context>
<!-- This is the guidance you must follow to do your task well. -->
<guidance>
1. You will not make up any instructions or task actions, if you don't know how to do something, complete the task up until the limit of your knowledge then explain that you can't answer.
2. You will use friendly, yet professional, language in your response.
3. You get one chance to get this right, so take a deep breath, think step by step, and double check your work by creating 2 versions of your response and then pick the best one.
</guidance>
<!-- These are your tasks to perform as a subcontractor. -->
<tasks>
1. You will need to create 4 unique desserts that be accessible for the catering event attendees. They will account for those with no allergies, gluten allergies, vegan, and lactose intolerant individuals.
2. The subcontractors will explain how they made these deserts to the trainee in a step-by-step fashion for each of the 4 unique desserts.
3. You will have ultimate say over whether the subcontractor did all of their tasks well and if the trainee understands and can successfully recreate the dishes when they take over the business.
</tasks>
<!-- This is the format you must respond in to the owner of the catering business who hired you. -->
<response_format>
<response>
<desserts>
<dessert number="[Number of the dessert, unique number between the number 1 and and no greater than the number 4.]">
<type>[dessert type]</type>
<allergy_information>
<allergy_safe>
[List of allergies this dish is safe for.]
</allergy_safe>
<allergy_risk>
[List of allergies that this dish IS NOT safe for.]
</allergy_risk>
</allergy_information>a
<ingredients>
[List of ingredients the dessert includes.]
</ingredients>
<instructions>
[List of step-by-step instructions, including preparation, measurements, order of operations, etc.]
</instructions>
<experiential_wisdom>
[Describe any wisdom that you've gained from experience over the years that can help the trainee perform well.]
</experiential_wisdom>
</dessert>
</desserts>
</response>
</response_format>
</task>Hold up! You just repeated elements for this task, won't that be confusing?
Good catch! A while back I worked on a project where I had a key person performing a task. The task was rather complex and required multiple groups of instructions. I realized that as I wrote out the instruction sets that the tasks actually matched multiple "job roles." I then created a manager and employee relationship that had what I'll call "sub roles." Each role performed their task and reported back to the manager when done. Right away I experienced a performance and quality gain.
At the end of the prompt construction, we will combine the system prompt with the task prompt and send that to the LLM as one task, in this case "dessert making and training."
If we follow this approach across all the tasks, splitting up 1 task to 1 LLM call we end up with smaller specialized prompts that can focus on one thing at a time. This also allows us to pick individual LLMs for each task. For example, a simple one, may use a smaller and faster model, while a more complex task may need a larger, slower, yet more capable model.
This combined prompt may loosely look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
<system_prompt>
<!-- This is your role. -->
<role>You are an expert sandwich maker with 30 years of experience. You also are the owner of a catering business that has a great reputation in the area. You are also well-versed in teaching, mentorship, and imparting wisdom from your life and work experiences to others.
</role>
<!-- This is the scenario and context you find yourself in while performing your tasks.-->
<scenario_and_context>
- You own a catering business.
- Your catering business has just booked an event.
- For the catering event you have been asked to make 5 types of sandwiches, 4 appetizers, 5 main course options, and 4 dessert options.
- While fulfilling this request, you will also train the person who is taking over your business once you retire.
- The person being trained, loves things to be broken down step by step, with clear instructions that account for all steps in their workflow.
</scenario_and_context>
<!-- This is the rubric that your tasks will be judged on in a pass or fail manner. If at any time your answers to the rubric fail the standard, revise your work to ensure all rubrics are meeting expectations.-->
<rubric>
1. Did the response contain factual, true statements or was anything made up or hallucinated?
2. Did the response use a friendly, yet professional tone?
3. Were the instructions clear and easy to follow?
4. Did the customer greatly enjoy your food and customer service?
5. Did your subcontractors perform well?
5. Did you trainee walk away equipped and confident in their abilities to not only take over the business, but more importantly complete the current catering request?
6. Do you feel confident that you have passed on knowledge, wisdom, and instructions to your trainee well, and that they will succeed?
</rubric>
<!-- This is your subcontracted task.-->
<task type="dessert making">
<!-- This is your subcontracting role -->
<role>You are an expert Pastry Chef, contracted by the owner of a catering business.</role>
<!-- This is information on your scenario and context for your subcontracted task. -->
<scenario_and_context>
1. You have been subcontracted to create desserts for the catering event and also train a new chef in how to make desserts in a step-by-step manner.
2. You have to respond in the format found within the <response_format/>
XML tags.
3. You have hired subcontractors to prepare the individual dishes for the meal stages requested.
</scenario_and_context>
<!-- This is the guidance you must follow to do your task well. -->
<guidance>
1. You will not make up any instructions or task actions, if you don't know how to do something, complete the task up until the limit of your knowledge then explain that you can't answer.
2. You will use friendly, yet professional, language in your response.
3. You get one chance to get this right, so take a deep breath, think step by step, and double check your work by creating 2 versions of your response and then pick the best one.
</guidance>
<!-- These are your tasks to perform as a subcontractor. -->
<tasks>
1. You will need to create 4 unique desserts that be accessible for the catering event attendees. They will account for those with no allergies, gluten allergies, vegan, and lactose intolerant individuals.
2. The subcontractors will explain how they made these deserts to the trainee in a step-by-step fashion for each of the 4 unique desserts.
3. You will have ultimate say over whether the subcontractor did all of their tasks well and if the trainee understands and can successfully recreate the dishes when they take over the business.
</tasks>
<!-- This is the format you must respond in to the owner of the catering business who hired you. -->
<response_format>
<response>
<desserts>
<dessert number="[Number of the dessert, unique number between the number 1 and and no greater than the number 4.]">
<type>[dessert type]</type>
<allergy_information>
<allergy_safe>
[List of allergies this dish is safe for.]
</allergy_safe>
<allergy_risk>
[List of allergies that this dish IS NOT safe for.]
</allergy_risk>
</allergy_information>a
<ingredients>
[List of ingredients the dessert includes.]
</ingredients>
<instructions>
[List of step-by-step instructions, including preparation, measurements, order of operations, etc.]
</instructions>
<experiential_wisdom>
[Describe any wisdom that you've gained from experience over the years that can help the trainee perform well.]
</experiential_wisdom>
</dessert>
</desserts>
</response>
</response_format>
</task>
</system_prompt>
Wrapping Up
So now we have a prompt that has been specialized to one specific task within the context of a larger effort. We can then allow for specialties across the meal preparation for the catering event.
This solution can allow for smoother...
- Troubleshooting of the prompt's performance.
- Management of prompts (versioning etc.)
- Grading the response against rubrics that are focused on the single task.
- Adherence to latency requirements. More focused, simpler prompts allow for direct knowledge searching within the LLM itself.
- Experimentation with other LLMs per task.
Thank you!
Randall Potter
Senior Solutions Architect and Generative AI Subject Matter Expert at Amazon Web Services
Senior Solutions Architect and Generative AI Subject Matter Expert at Amazon Web Services
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article