OpenAI made waves in the world of artificial intelligence development with the introduction of an array of new features and enhancements during its SF DevDay 2024 on October 1. The event, held in San Francisco, drew considerable attention from technology enthusiasts and developers, eager to explore advancements designed to expand the utility and efficiency of AI applications.
One of the standout introductions was the Real-Time API with function calling, designed to enable persistent WebSocket connections. This feature allows for authentic real-time voice interactions, making it a promising tool for applications necessitating immediate responses such as virtual assistants and real-time translation services. The API is structured to exchange JSON-formatted events capturing elements such as text, audio, and even function calls, alongside handling multimodal outputs simultaneously. Priced competitively at approximately $0.30 per minute, this service extends beyond traditional chat functionalities, facilitating AI-initiated actions.
In a demonstration that highlighted the API's capabilities, a travel agent application showcased the function calling feature, illustrating how AI could interact with external tools and databases. This effectively positions AI as a dynamic intermediary, capable of performing actions beyond static pre-trained responses. OpenAI noted the importance of incorporating more user control over safety settings and indicated that a "safety API" could be in development to address these concerns.
Additionally, the DevDay event provided a glimpse into OpenAI’s O1 model through a coding demo. This model is positioned as a solution not just for code generation but for assisting in understanding and architecting software solutions. In a presentation, a developer demonstrated the model's potential by describing a mobile application to O1, which then helped produce the necessary coding framework. OpenAI conceded that while metrics such as Sweebench are beneficial for measuring code accuracy, they may not fully reflect the model's capabilities in practical scenarios, particularly in UI development.
OpenAI also announced enhancements to its vision model's fine-tuning capabilities, allowing developers to customize these models for specific tasks. This process involves modifying hyperparameters, supported by integration with Weights and Biases, a toolset designed for tracking and evaluating fine-tuning jobs. OpenAI emphasised its commitment to safety, noting that automated safety evaluations are carried out on fine-tuned models to ensure compliance with usage policies.
To make their APIs more accessible financially, OpenAI has rolled out a model distillation API and new evaluation tools. Model distillation focuses on creating lighter models while aiming to maintain performance, particularly crucial for deployment in resource-limited environments. Additionally, prompt caching was introduced as a measure to reduce latency through reusing previously processed prompts. For optimal caching, developers are advised to format prompts with static content first, followed by dynamic content—a structural strategy designed to maximize cache efficiency. Commenting on the improvement, Shawn Wang noted that while the discount offered by OpenAI's prompt caching may not rival others like Gemini and Anthropic, it stands out by requiring no code changes.
Further events are slated, with upcoming sessions scheduled in London on October 30 and Singapore on November 21. These will offer further opportunities for developers to delve into the newly unveiled technological offerings from OpenAI, with detailed documentation available to guide them through the innovations presented.
Source: Noah Wire Services