Google introduces photo support for AI search mode, use it like this

Google built the multimodal features of the Lens technology in order to boost the strength of its AI-driven search feature, the AI mode. Users will soon be able to search using both text and images, as opposed to images alone, providing them with a richer and contextualized search environment. According to Google, the early adopters of the AI mode received favorable feedback for the intuitive interface, quick response, and capacity to recognize complicated and detailed questions.

AI mode is one of the experimental and opt-in options in the Google Search Labs. It takes search to an AI-based platform that makes it possible for people to pose new questions and investigate matters in greater detail. For now, the feature comes with a subscription to Google One AI Premium. According to Google, people are using AI mode for exploratory and open-ended questions, as well as complex tasks like comparing two products, finding "how to" guides, and trip planning.

Users can now ask questions about a photo by taking a photo or uploading an image. The AI ​​mode will then dig deep into the image and provide complete, detailed answers - including related links so users can learn more.

This integration is based on Google's custom-built Gemini model, which has the ability to recognize objects in an image as well as understand the entire scene - such as the relationships between objects, their material, color, shape and arrangement, etc.

While the Lens technology accurately identifies every element in an image, the AI ​​mode goes a step further and automatically generates a variety of questions about that image. This process is called the "Query Fan-out Technique", which allows the AI ​​mode to provide more depth and variety of information than traditional text-based search.