NexusAI-Client

Core Features

Deep dive into fallback routing, streaming, multimodal vision, structured JSON, and budget inspection.

Core Features ⚡

NexusAI-Client gives you a standardized, high-performance toolkit for building production AI applications.


1. Zero-Cost-First Smart Fallback

Automatic progression from 100% free models to ultra-low-cost commercial models. If a free provider throws HTTP 429 (Rate Limit), times out, or experiences a network outage, NexusAI-Client transparently tries the next provider in the chain.

Automatic Discovery:

import asyncio
from nexusai_client import AIGateway

async def main():
    # Detects active keys in .env and routes: Free Tiers -> Paid Backups
    async with AIGateway.auto_fallback() as client:
        res = await client.generate_text("Explain transformer attention mechanisms.")
        print(f"Generated via: [{res.provider}] / Model: [{res.model}]")
        print(res.text)

if __name__ == "__main__":
    asyncio.run(main())

Custom Priority Chain:

# Custom explicit chain: Cerebras -> Groq -> Gemini Free -> DeepSeek
custom_chain = ["cerebras", "groq", "gemini_free", "deepseek"]

async with AIGateway.with_fallback(custom_chain, timeout=20.0) as client:
    res = await client.generate_text("Summarize the news.")
    print(f"[{res.provider}] {res.text}")

2. Real-Time Token Streaming (SSE)

Stream generated chunks asynchronously using Python async for:

import asyncio
from nexusai_client import AIGateway

async def main():
    async with AIGateway("groq") as client:
        print("Streaming token-by-token:")
        async for chunk in client.stream_text("Write a short story about artificial intelligence."):
            print(chunk, end="", flush=True)

if __name__ == "__main__":
    asyncio.run(main())

[!NOTE] Streaming is also fully supported when using FallbackGateway (AIGateway.auto_fallback())!


3. Multimodal Vision Analysis

Analyze screenshots, charts, PDFs, or photos. You can supply:

  • A local file path (Path or str)
  • A remote image URL (https://...)
  • Raw image bytes (bytes)
import asyncio
from nexusai_client import AIGateway

async def main():
    # Automatically selects the best Vision model (Gemini 2.5 Flash, Llama 3.2 Vision, Qwen 3.8 Vision, Aya Vision, Pixtral)
    async with AIGateway.auto_fallback_vision() as client:
        res = await client.analyze_image(
            prompt="Extract the invoice line items, tax rate, and total amount as JSON.",
            image="invoice_sample.png",
            json_mode=True,
        )
        print(f"Vision provider: {res.provider}")
        print(res.text)

if __name__ == "__main__":
    asyncio.run(main())

4. Guaranteed Structured JSON Mode

Standardize JSON outputs across all providers by setting json_mode=True:

import asyncio
import json
from nexusai_client import AIGateway

async def main():
    async with AIGateway("groq") as client:
        res = await client.generate_text(
            prompt="Extract user profile from: 'John Doe, 32, Lead AI Architect at Zurich'",
            system_prompt="Output schema: {'name': str, 'age': int, 'role': str, 'city': str}",
            json_mode=True,
        )
        data = json.loads(res.text)
        print("Parsed JSON data:", data)

if __name__ == "__main__":
    asyncio.run(main())

5. Live Balances & Quota Monitoring

Inspect account credits and quota limits in real-time:

import asyncio
from nexusai_client import AIGateway

async def main():
    # 1. Single provider inspection
    async with AIGateway("deepseek") as client:
        account = await client.get_account_info()
        print(account.format_summary())
        # Example output: "Solde restant: $4.99 | (Offert: $0.00) | Consommé: $0.0120"

    # 2. Parallel inspection across all configured providers
    all_accounts = await AIGateway.get_all_account_infos()
    for prov_name, info in all_accounts.items():
        print(f"[{prov_name.upper()}]: {info.format_summary()}")

if __name__ == "__main__":
    asyncio.run(main())

6. Model Catalog Discovery & Dynamic Pricing

Query live model catalogs and per-million token pricing across all connected providers:

import asyncio
from nexusai_client import AIGateway

async def main():
    # List free models across OpenRouter
    async with AIGateway("openrouter") as client:
        models = await client.list_models(free_only=True)
        print(f"Found {len(models)} free models on OpenRouter:")
        for m in models[:5]:
            print(f" - {m.id} (Context: {m.context_length} tokens)")

    # Parallel discovery across all providers
    catalog = await AIGateway.list_all_available_models(free_only=True)
    for prov, m_list in catalog.items():
        print(f"Provider [{prov}] offers {len(m_list)} free models.")

if __name__ == "__main__":
    asyncio.run(main())

7. Universal Tool Calling & Function Calling

Equip AI models with external tools, database queries, calculators, or API integrations.

NexusAI-Client provides a universal standard schema that works identically across all supported providers. Under the hood, it transparently translates schemas and payloads into provider-native formats:

  • OpenAI-Compatible Providers (Groq, Cerebras, Mistral, DeepSeek, Nvidia NIM, OpenRouter, OrcaRouter): standard tools array with type: "function".
  • Google Gemini REST API (gemini_free, gemini_pro): automatically translated to Gemini's functionDeclarations and functionCall / functionResponse parts.
  • Cohere V2 REST API (cohere): translated to Cohere's V2 tool calling format.

Defining Tools:

from nexusai_client import FunctionDefinition, ToolDefinition

weather_tool = ToolDefinition(
    function=FunctionDefinition(
        name="get_current_weather",
        description="Retrieve the current temperature and conditions for a given city.",
        parameters={
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and state/country, e.g. 'Tokyo, Japan'",
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Temperature unit",
                },
            },
            "required": ["location"],
        },
    )
)

Executing a Tool-Aware Chat Request:

import asyncio
from nexusai_client import AIGateway, ChatMessage

async def main():
    # Tools work seamlessly across single providers and auto-fallback chains!
    async with AIGateway.auto_fallback() as client:
        messages = [
            ChatMessage(role="user", content="What is the weather like in Tokyo right now?")
        ]
        
        response = await client.chat(messages=messages, tools=[weather_tool])

        if response.has_tool_calls:
            for call in response.tool_calls:
                print(f"🔧 Tool ID       : {call.id}")
                print(f"📌 Function Name : {call.name}")
                print(f"📦 Arguments     : {call.arguments}")
                # e.g. Arguments: {'location': 'Tokyo, Japan', 'unit': 'celsius'}
        else:
            print("💬 Normal Text Reply:", response.text)

if __name__ == "__main__":
    asyncio.run(main())

[!TIP] For building complete multi-turn autonomous agent loops that execute local Python functions and feed outputs back into the model, check out the Autonomous AI Agent Recipe.


8. Intelligent Gemini Free Model Rotation (Auto 429 Quota Failover)

Google AI Studio Free Tier offers world-class models (gemini-3.5-flash-lite, gemini-3.7-flash, etc.) with massive context windows (up to 1M tokens) at zero cost. However, Google enforces strict per-model rate limits and daily quota pools:

  • Flash-Lite Models: ~500 Requests/day (RPD), 15 RPM, 250k TPM
  • Flash Models: ~20 Requests/day (RPD), 15 RPM, 1M TPM
  • Gemma Open Models: ~14,400 Requests/day (RPD), 30 RPM, 30k TPM

When a single model hits its quota limit (HTTP 429 RESOURCE_EXHAUSTED), traditional SDKs fail immediately. NexusAI-Client solves this natively with a 2-Tier Fallback Hierarchy:

 ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
 │                                   LEVEL 1: INTRA-PROVIDER SMART MODEL ROTATION                             │
 │                                                                                                             │
 │  1. gemini-3.5-flash-lite (500 RPD) ──► 2. gemini-3.1-flash-lite (500 RPD) ──► 3. gemini-flash-lite-latest   │
 │                                                                                                             │
 │       ▼ (If 429 Quota Exceeded)                                                                             │
 │  4. gemini-3.7-flash (20 RPD)       ──► 5. gemini-3.6-flash (20 RPD)       ──► 6. gemini-3.5-flash (20 RPD)   │
 │                                                                                                             │
 │       ▼ (If 429 Quota Exceeded)                                                                             │
 │  7. gemini-flash-latest             ──► 8. gemini-2.5-flash-lite (500 RPD) ──► 9. gemini-2.5-flash (20 RPD)   │
 │                                                                                                             │
 │       ▼ (If 429 Quota Exceeded)                                                                             │
 │  10. gemma-4-31b-it (14.4k RPD)     ──► 11. gemma-4-26b-a4b-it (14.4k RPD)                                  │
 └──────────────────────────────────────────────────────┬──────────────────────────────────────────────────────┘
                                                        │ (Only if ALL 11 Gemini models exhausted)

 ┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
 │                               LEVEL 2: INTER-PROVIDER AUTO-FALLBACK FAILOVER                                │
 │   Groq LPU ──► Cerebras CS-3 ──► Nvidia NIM ──► OrcaRouter ──► Mistral ──► Cohere ──► OpenRouter ──► DeepSeek│
 └─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Complete 11-Model Cascade Chain

StepModel IdentifierQuotas (Free Tier)Context WindowPrimary Use Case
1gemini-3.5-flash-lite (Default)500 RPD | 15 RPM | 250k TPM1,048,576 tokensUltra-fast default general queries & tool calling
2gemini-3.1-flash-lite500 RPD | 15 RPM | 250k TPM1,048,576 tokensFast secondary lightweight failover
3gemini-flash-lite-latest500 RPD | 15 RPM | 250k TPM1,048,576 tokensLatest stable flash-lite pointer alias
4gemini-3.7-flash20 RPD | 15 RPM | 1,000,000 TPM1,048,576 tokensHigh-reasoning & complex logic tasks
5gemini-3.6-flash20 RPD | 15 RPM | 1,000,000 TPM1,048,576 tokensAdvanced multimodal & code synthesis
6gemini-3.5-flash20 RPD | 15 RPM | 1,000,000 TPM1,048,576 tokensGeneral multimodal & vision failover
7gemini-flash-latest20 RPD | 15 RPM | 1,000,000 TPM1,048,576 tokensLatest stable flash pointer alias
8gemini-2.5-flash-lite500 RPD | 15 RPM | 250k TPM1,048,576 tokensPrevious-generation fast fallback
9gemini-2.5-flash20 RPD | 15 RPM | 1,000,000 TPM1,048,576 tokensPrevious-generation robust fallback
10gemma-4-31b-it14,400 RPD | 30 RPM | 30k TPM131,072 tokensHigh-volume open-weight model with massive RPD
11gemma-4-26b-a4b-it14,400 RPD | 30 RPM | 30k TPM131,072 tokensFinal emergency high-RPD free tier model

Multimodal Vision Rotation Sequence

For analyze_image(), the rotation automatically restricts itself to the 7 vision-capable Gemini models: gemini-3.5-flash-litegemini-3.1-flash-litegemini-3.7-flashgemini-3.6-flashgemini-3.5-flashgemini-2.5-flash-litegemini-2.5-flash.

Stateful Cooldown & Zero Latency Penalty

  • Automatic Cooldowns: When a model returns HTTP 429, it is immediately placed in cooldown (cooldown_seconds=60.0 by default). Deprecated/404 models are cooled down for 1 hour.
  • Zero Retrial Overhead: Subsequent requests during the same process lifecycle instantly skip cooled-down models, routing directly to the first available operational model with 0ms penalty.
  • Real-Time Visibility: Inspect live model statuses and cooldown timers with get_account_info().

Code Examples

import asyncio
from nexusai_client import AIGateway

async def main():
    # 1. Zero configuration auto-failover
    async with AIGateway("gemini_free") as client:
        res = await client.generate_text("Explain quantum entanglement simply.")
        print(f"✅ Served by model [{res.model}] (Provider: {res.provider}):\n{res.text}")

        # 2. Live rotation status and cooldown diagnostics
        info = await client.get_account_info()
        print(f"Active Model: {info.extra_details['current_active_model']}")
        print(f"Cooldown Status: {info.extra_details['models_in_cooldown']}")

if __name__ == "__main__":
    asyncio.run(main())

9. 100% Free Multi-Provider Fallback (auto_fallback_free)

Build zero-cost resilient workflows by cascading across all free providers present in your .env:

import asyncio
from nexusai_client import AIGateway

async def main():
    # Cascade: Gemini Free (11 models) -> Groq LPU -> Cerebras CS-3 -> Nvidia NIM -> OrcaRouter -> Mistral -> Cohere -> OpenRouter
    async with AIGateway.auto_fallback_free() as client:
        res = await client.generate_text("Write a concise summary of AI agents architecture.")
        print(f"✅ Served by [{res.provider} / {res.model}]:\n{res.text}")

if __name__ == "__main__":
    asyncio.run(main())

On this page