Fixing Gemini Tool Calls: Unsupported Schema Keywords Bug

by Editorial Team 58 views
Iklan Headers

Hey guys! Ever run into a snag when trying to get those cool tool calls working with Google's Gemini models? You're not alone. I've been wrestling with a frustrating issue where the Gemini API throws a fit because of some unsupported keywords in the tool schemas. Let's dive deep into this bug, understand the root cause, and figure out how to squash it. I'll break down the problem, the suggested fix, and how you can get your tools running smoothly. This will cover everything from identifying the culprit to implementing the solution. Buckle up!

The Bug: Gemini Tool Calls Crashing

So, the main issue, as you might have guessed, is that when you try to use tool calls with Google/Gemini/Antigravity models, things go south. Specifically, you get an error that screams "google tool schema has unsupported keywords." This happens because the Gemini API, bless its digital heart, doesn't play nice with all the fancy schema keywords we might be using in our tool definitions. It's like trying to get a picky eater to enjoy a gourmet meal – some ingredients just aren't welcome.

The error message itself gives us a clue: {"index":7,"tool":"browser","violations":["browser.parameters.properties.format"],"violationCount":1}. This snippet tells us that the "browser" tool, at index 7, has a problem with the format property in its schema. The core problem is that the Gemini API has limitations on which format types it supports. For example, it only officially supports format: "enum" or format: "date-time" for STRING types. Any other format values, like format: "uri", will trigger an INVALID_ARGUMENT error, and boom – your tool call fails.

Impacted Tools

  • Browser Tool: This is a common one, and a frequent offender. The format property is often used in the browser tool schema, and it's a prime target for this error.
  • Canvas Tool: Another tool that can get tripped up by unsupported schema keywords. Similar to the browser tool, it might use format properties that the Gemini API doesn't like.

Root Cause: Unsupported Schema Keywords

Alright, let's get down to the nitty-gritty and find out what's really happening under the hood. The problem stems from the fact that Google's Gemini models have limitations when it comes to the types of schema keywords they support within tool definitions. Some keywords, like format, additionalProperties, and $ref, aren't fully compatible and can cause the API to throw an error.

The code identifies the unsupported keywords, which is good, with GOOGLE_SCHEMA_UNSUPPORTED_KEYWORDS set. The findUnsupportedSchemaKeywords() function detects the violations, but there's no way of fixing it yet. The problem isn't that we don't know what's wrong; it's that we don't have the tools to fix it before sending the data to the API. The logToolSchemasForGoogle() function only logs the issue, which is helpful for debugging, but doesn't actually solve the problem.

The missing piece is a function that can strip out these unsupported keywords from the tool schemas before they are sent to the Google API. We need a way to go through the schema definitions and remove or modify those keywords that the Gemini API doesn't support. This involves recursively going through the schema and deleting certain parts of it. Think of it like pre-processing your data to fit the exact requirements of the API.

Gemini API Limitations

  • The Gemini API has very specific rules about what it accepts in the format property, and it does not allow all of the available format types. It's important to know the boundaries of the API.
  • The API's strictness with schema definitions is why we see this INVALID_ARGUMENT error. The error is a direct result of these unsupported keywords.

Suggested Fix: Strip Unsupported Keywords

So, how do we solve this? The suggested fix is pretty straightforward: add a stripUnsupportedSchemaKeywords() function to clean up the tool schemas before they're sent to the Google API. This function would recursively go through the schema and remove any of the keywords listed in GOOGLE_SCHEMA_UNSUPPORTED_KEYWORDS. It's like doing a deep clean on your data to remove all the bits that the API doesn't like.

Here’s a breakdown of what the fix would look like:

  1. Implement stripUnsupportedSchemaKeywords(): This function would take a tool schema as input and recursively go through it, identifying and removing the offending keywords. This function would need to handle nested schemas and correctly remove those properties.
  2. Call the function: Before sending the tool definitions to the Google API, you'd call stripUnsupportedSchemaKeywords() on each tool schema.

This fix aligns with existing logic used for session history, which already has similar cleanup mechanisms. We're essentially applying the same principle to tool schemas.

Reproduction

Want to see this bug in action? Here's how to reproduce it:

  1. Set up Models: Use clawdbot models and configure them to use google-antigravity/claude-opus-4-5.
  2. Try Tools: Attempt to use either the browser or canvas tools. These tools are known to use schemas with unsupported keywords.

If you're using these models and tools, you'll likely run into the error described above. It's a pretty easy bug to trigger once you have the right setup!

Workaround

If you’re stuck and can't wait for a fix, there is a workaround: use Anthropic or OpenAI models instead. These models may have different, or more relaxed, schema requirements, and won't trigger the same errors. Of course, this means you're relying on a different model provider, which may have implications for your project.

Conclusion: Keeping Your Tools Running Smoothly

This bug can be a headache, but with the right fix, it's definitely manageable. The core of the solution is to clean up your tool schemas before sending them to the Gemini API. This involves identifying and removing unsupported keywords, so the API doesn't throw INVALID_ARGUMENT errors. Implementing a stripUnsupportedSchemaKeywords() function is the key to resolving this issue, and the reproduction steps make it easy for you to test and confirm the fix. By taking these steps, you can get your tool calls up and running with Gemini, letting you get back to building the future!

I hope this helps you guys troubleshoot and resolve this issue. Happy coding!