sayed99 commited on
Commit
e104f68
·
1 Parent(s): 48d2716

teached the agent to generate and caption images(connect things together)

Browse files
Files changed (2) hide show
  1. prompts.yaml +21 -4
  2. tools/image_generation.py +3 -6
prompts.yaml CHANGED
@@ -21,15 +21,32 @@
21
  ```<end_code>
22
  Observation: "The oldest person in the document is John Doe, a 55 year old lumberjack living in Newfoundland."
23
 
24
- Thought: I will now generate an image showcasing the oldest person and save it in generations dir.
25
  Code:
26
  ```py
27
- import uuid
28
  image, img_path = image_generator("A portrait of John Doe, a 55-year-old man living in Canada.")
 
29
  final_answer(image)
30
- final_answer(img_path)
31
  ```<end_code>
32
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  ---
34
  Task: "What is the result of the following operation: 5 + 3 + 1294.678?"
35
 
 
21
  ```<end_code>
22
  Observation: "The oldest person in the document is John Doe, a 55 year old lumberjack living in Newfoundland."
23
 
24
+ Thought: I will now generate an image showcasing the oldest person and it will be saved in generations dir.
25
  Code:
26
  ```py
 
27
  image, img_path = image_generator("A portrait of John Doe, a 55-year-old man living in Canada.")
28
+ print("The image has been generated and saved in ", img_path)
29
  final_answer(image)
 
30
  ```<end_code>
31
+
32
+ ---
33
+ Task: "Generate image for a person walking on the moon, and then caption it"
34
+ Thought: First I will use the tool `image_generator` to generate an image of a person walking on the moon.
35
+ Code:
36
+ ```py
37
+ image, img_path = image_generator("A person walking on the moon.")
38
+ print("The image has been generated and saved in ", img_path)
39
+ final_answer(image)
40
+ ```<end_code>
41
+ Observation: "The image has been generated and saved in the 'generations' directory."
42
+
43
+ Thought: Now I will use the tool `image_captioning` to caption the generated image.
44
+ Code:
45
+ ```py
46
+ caption = image_captioning(image)
47
+ final_answer(caption)
48
+ ```<end_code>
49
+
50
  ---
51
  Task: "What is the result of the following operation: 5 + 3 + 1294.678?"
52
 
tools/image_generation.py CHANGED
@@ -11,15 +11,12 @@ from PIL import Image
11
  class ImageGenerationTool(Tool):
12
  name = "image_generation"
13
  description = """
14
- Generates an image based on the given prompt and saves it locally.
15
-
16
  Args:
17
  prompt (str): The prompt for image generation.
18
-
19
  Returns:
20
- tuple: A tuple containing:
21
- - Image.Image: The generated image.
22
- - pathlib.Path: The file path where the image was saved.
23
  """
24
  inputs = {'prompt': {'type': 'string',
25
  'description': 'The prompt for image generation.'}}
 
11
  class ImageGenerationTool(Tool):
12
  name = "image_generation"
13
  description = """
14
+ Generates an image based on the given prompt and saves it locally at generations dir.
 
15
  Args:
16
  prompt (str): The prompt for image generation.
 
17
  Returns:
18
+ - Image.Image: The generated image.
19
+ - str: The path where the image is saved.
 
20
  """
21
  inputs = {'prompt': {'type': 'string',
22
  'description': 'The prompt for image generation.'}}