In a recent exploration of cutting-edge technology, AI researcher Simon Willison shared an intriguing method of data extraction that could signal a shift in how artificial intelligence interacts with digital information. Faced with the task of consolidating numeric data from a set of disparate emails related to cloud service charges, Willison employed an innovative technique he terms "video scraping". This technique involves utilising AI to analyse video screen recordings to extract and organise information efficiently.
Willison outlined his process and findings in an insightful blog post. To tackle his data aggregation challenge, he captured a 35-second video of himself scrolling through the emails containing the necessary payment details. This visual data was then fed into Google's AI Studio, a platform that lets users experiment with various iterations of Google's AI models, specifically the Gemini 1.5 Pro and Gemini 1.5 Flash.
Once the video was processed, Willison tasked Gemini with parsing the video to retrieve and organise the price data, including dates and dollar amounts, into JSON format (JavaScript Object Notation). This data format is known for its ease of conversion into other formats and uses, such as CSV (comma-separated values), which is particularly handy for spreadsheet applications.
The AI's successful extraction of all relevant data into a structured format impressed Willison, notably because of the model's precision and efficiency. Moreover, the operation proved to be astonishingly cost-effective. The use of the AI model, specifically the Gemini 1.5 Flash 002, required 11,018 tokens, translating into a cost of less than one-tenth of a cent. Remarkably, this process incurred no actual expenses for Willison, as Google AI Studio currently supports free usage for certain experimental activities.
This experiment underscores the practical capabilities of AI in potentially transforming mundane administrative tasks into streamlined processes. By demonstrating that minimal resources can be utilised for complex data extraction tasks through video analysis, Willison's work could pave the way for further advancements in AI applications aimed at desktop and data management tasks. The ease of use and cost-efficiency displayed in Willison's experiment suggests potential future developments where AI might seamlessly integrate with daily digital tasks, enhancing productivity without incurring prohibitive costs.
Source: Noah Wire Services