Core Features
Deep dive into fallback routing, streaming, multimodal vision, structured JSON, and budget inspection.
Core Features ⚡
NexusAI-Client gives you a standardized, high-performance toolkit for building production AI applications.
1. Zero-Cost-First Smart Fallback
Automatic progression from 100% free models to ultra-low-cost commercial models. If a free provider throws HTTP 429 (Rate Limit), times out, or experiences a network outage, NexusAI-Client transparently tries the next provider in the chain.
Automatic Discovery:
import asyncio
from nexusai_client import AIGateway
async def main():
# Detects active keys in .env and routes: Free Tiers -> Paid Backups
async with AIGateway.auto_fallback() as client:
res = await client.generate_text("Explain transformer attention mechanisms.")
print(f"Generated via: [{res.provider}] / Model: [{res.model}]")
print(res.text)
if __name__ == "__main__":
asyncio.run(main())Custom Priority Chain:
# Custom explicit chain: Cerebras -> Groq -> Gemini Free -> DeepSeek
custom_chain = ["cerebras", "groq", "gemini_free", "deepseek"]
async with AIGateway.with_fallback(custom_chain, timeout=20.0) as client:
res = await client.generate_text("Summarize the news.")
print(f"[{res.provider}] {res.text}")2. Real-Time Token Streaming (SSE)
Stream generated chunks asynchronously using Python async for:
import asyncio
from nexusai_client import AIGateway
async def main():
async with AIGateway("groq") as client:
print("Streaming token-by-token:")
async for chunk in client.stream_text("Write a short story about artificial intelligence."):
print(chunk, end="", flush=True)
if __name__ == "__main__":
asyncio.run(main())[!NOTE] Streaming is also fully supported when using
FallbackGateway(AIGateway.auto_fallback())!
3. Multimodal Vision Analysis
Analyze screenshots, charts, PDFs, or photos. You can supply:
- A local file path (
Pathorstr) - A remote image URL (
https://...) - Raw image bytes (
bytes)
import asyncio
from nexusai_client import AIGateway
async def main():
# Automatically selects the best Vision model (Gemini 2.5 Flash, Llama 3.2 Vision, Qwen 3.8 Vision, Aya Vision, Pixtral)
async with AIGateway.auto_fallback_vision() as client:
res = await client.analyze_image(
prompt="Extract the invoice line items, tax rate, and total amount as JSON.",
image="invoice_sample.png",
json_mode=True,
)
print(f"Vision provider: {res.provider}")
print(res.text)
if __name__ == "__main__":
asyncio.run(main())4. Guaranteed Structured JSON Mode
Standardize JSON outputs across all providers by setting json_mode=True:
import asyncio
import json
from nexusai_client import AIGateway
async def main():
async with AIGateway("groq") as client:
res = await client.generate_text(
prompt="Extract user profile from: 'John Doe, 32, Lead AI Architect at Zurich'",
system_prompt="Output schema: {'name': str, 'age': int, 'role': str, 'city': str}",
json_mode=True,
)
data = json.loads(res.text)
print("Parsed JSON data:", data)
if __name__ == "__main__":
asyncio.run(main())5. Live Balances & Quota Monitoring
Inspect account credits and quota limits in real-time:
import asyncio
from nexusai_client import AIGateway
async def main():
# 1. Single provider inspection
async with AIGateway("deepseek") as client:
account = await client.get_account_info()
print(account.format_summary())
# Example output: "Solde restant: $4.99 | (Offert: $0.00) | Consommé: $0.0120"
# 2. Parallel inspection across all configured providers
all_accounts = await AIGateway.get_all_account_infos()
for prov_name, info in all_accounts.items():
print(f"[{prov_name.upper()}]: {info.format_summary()}")
if __name__ == "__main__":
asyncio.run(main())6. Model Catalog Discovery & Dynamic Pricing
Query live model catalogs and per-million token pricing across all connected providers:
import asyncio
from nexusai_client import AIGateway
async def main():
# List free models across OpenRouter
async with AIGateway("openrouter") as client:
models = await client.list_models(free_only=True)
print(f"Found {len(models)} free models on OpenRouter:")
for m in models[:5]:
print(f" - {m.id} (Context: {m.context_length} tokens)")
# Parallel discovery across all providers
catalog = await AIGateway.list_all_available_models(free_only=True)
for prov, m_list in catalog.items():
print(f"Provider [{prov}] offers {len(m_list)} free models.")
if __name__ == "__main__":
asyncio.run(main())7. Universal Tool Calling & Function Calling
Equip AI models with external tools, database queries, calculators, or API integrations.
NexusAI-Client provides a universal standard schema that works identically across all supported providers. Under the hood, it transparently translates schemas and payloads into provider-native formats:
- OpenAI-Compatible Providers (Groq, Cerebras, Mistral, DeepSeek, Nvidia NIM, OpenRouter, OrcaRouter): standard
toolsarray withtype: "function". - Google Gemini REST API (
gemini_free,gemini_pro): automatically translated to Gemini'sfunctionDeclarationsandfunctionCall/functionResponseparts. - Cohere V2 REST API (
cohere): translated to Cohere's V2 tool calling format.
Defining Tools:
from nexusai_client import FunctionDefinition, ToolDefinition
weather_tool = ToolDefinition(
function=FunctionDefinition(
name="get_current_weather",
description="Retrieve the current temperature and conditions for a given city.",
parameters={
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City and state/country, e.g. 'Tokyo, Japan'",
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit",
},
},
"required": ["location"],
},
)
)Executing a Tool-Aware Chat Request:
import asyncio
from nexusai_client import AIGateway, ChatMessage
async def main():
# Tools work seamlessly across single providers and auto-fallback chains!
async with AIGateway.auto_fallback() as client:
messages = [
ChatMessage(role="user", content="What is the weather like in Tokyo right now?")
]
response = await client.chat(messages=messages, tools=[weather_tool])
if response.has_tool_calls:
for call in response.tool_calls:
print(f"🔧 Tool ID : {call.id}")
print(f"📌 Function Name : {call.name}")
print(f"📦 Arguments : {call.arguments}")
# e.g. Arguments: {'location': 'Tokyo, Japan', 'unit': 'celsius'}
else:
print("💬 Normal Text Reply:", response.text)
if __name__ == "__main__":
asyncio.run(main())[!TIP] For building complete multi-turn autonomous agent loops that execute local Python functions and feed outputs back into the model, check out the Autonomous AI Agent Recipe.
8. Intelligent Gemini Free Model Rotation (Auto 429 Quota Failover)
Google AI Studio Free Tier offers world-class models (gemini-3.5-flash-lite, gemini-3.7-flash, etc.) with massive context windows (up to 1M tokens) at zero cost. However, Google enforces strict per-model rate limits and daily quota pools:
- Flash-Lite Models: ~500 Requests/day (RPD), 15 RPM, 250k TPM
- Flash Models: ~20 Requests/day (RPD), 15 RPM, 1M TPM
- Gemma Open Models: ~14,400 Requests/day (RPD), 30 RPM, 30k TPM
When a single model hits its quota limit (HTTP 429 RESOURCE_EXHAUSTED), traditional SDKs fail immediately. NexusAI-Client solves this natively with a 2-Tier Fallback Hierarchy:
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ LEVEL 1: INTRA-PROVIDER SMART MODEL ROTATION │
│ │
│ 1. gemini-3.5-flash-lite (500 RPD) ──► 2. gemini-3.1-flash-lite (500 RPD) ──► 3. gemini-flash-lite-latest │
│ │
│ ▼ (If 429 Quota Exceeded) │
│ 4. gemini-3.7-flash (20 RPD) ──► 5. gemini-3.6-flash (20 RPD) ──► 6. gemini-3.5-flash (20 RPD) │
│ │
│ ▼ (If 429 Quota Exceeded) │
│ 7. gemini-flash-latest ──► 8. gemini-2.5-flash-lite (500 RPD) ──► 9. gemini-2.5-flash (20 RPD) │
│ │
│ ▼ (If 429 Quota Exceeded) │
│ 10. gemma-4-31b-it (14.4k RPD) ──► 11. gemma-4-26b-a4b-it (14.4k RPD) │
└──────────────────────────────────────────────────────┬──────────────────────────────────────────────────────┘
│ (Only if ALL 11 Gemini models exhausted)
▼
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ LEVEL 2: INTER-PROVIDER AUTO-FALLBACK FAILOVER │
│ Groq LPU ──► Cerebras CS-3 ──► Nvidia NIM ──► OrcaRouter ──► Mistral ──► Cohere ──► OpenRouter ──► DeepSeek│
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘Complete 11-Model Cascade Chain
| Step | Model Identifier | Quotas (Free Tier) | Context Window | Primary Use Case |
|---|---|---|---|---|
| 1 | gemini-3.5-flash-lite (Default) | 500 RPD | 15 RPM | 250k TPM | 1,048,576 tokens | Ultra-fast default general queries & tool calling |
| 2 | gemini-3.1-flash-lite | 500 RPD | 15 RPM | 250k TPM | 1,048,576 tokens | Fast secondary lightweight failover |
| 3 | gemini-flash-lite-latest | 500 RPD | 15 RPM | 250k TPM | 1,048,576 tokens | Latest stable flash-lite pointer alias |
| 4 | gemini-3.7-flash | 20 RPD | 15 RPM | 1,000,000 TPM | 1,048,576 tokens | High-reasoning & complex logic tasks |
| 5 | gemini-3.6-flash | 20 RPD | 15 RPM | 1,000,000 TPM | 1,048,576 tokens | Advanced multimodal & code synthesis |
| 6 | gemini-3.5-flash | 20 RPD | 15 RPM | 1,000,000 TPM | 1,048,576 tokens | General multimodal & vision failover |
| 7 | gemini-flash-latest | 20 RPD | 15 RPM | 1,000,000 TPM | 1,048,576 tokens | Latest stable flash pointer alias |
| 8 | gemini-2.5-flash-lite | 500 RPD | 15 RPM | 250k TPM | 1,048,576 tokens | Previous-generation fast fallback |
| 9 | gemini-2.5-flash | 20 RPD | 15 RPM | 1,000,000 TPM | 1,048,576 tokens | Previous-generation robust fallback |
| 10 | gemma-4-31b-it | 14,400 RPD | 30 RPM | 30k TPM | 131,072 tokens | High-volume open-weight model with massive RPD |
| 11 | gemma-4-26b-a4b-it | 14,400 RPD | 30 RPM | 30k TPM | 131,072 tokens | Final emergency high-RPD free tier model |
Multimodal Vision Rotation Sequence
For analyze_image(), the rotation automatically restricts itself to the 7 vision-capable Gemini models:
gemini-3.5-flash-lite ➔ gemini-3.1-flash-lite ➔ gemini-3.7-flash ➔ gemini-3.6-flash ➔ gemini-3.5-flash ➔ gemini-2.5-flash-lite ➔ gemini-2.5-flash.
Stateful Cooldown & Zero Latency Penalty
- Automatic Cooldowns: When a model returns
HTTP 429, it is immediately placed in cooldown (cooldown_seconds=60.0by default). Deprecated/404 models are cooled down for 1 hour. - Zero Retrial Overhead: Subsequent requests during the same process lifecycle instantly skip cooled-down models, routing directly to the first available operational model with 0ms penalty.
- Real-Time Visibility: Inspect live model statuses and cooldown timers with
get_account_info().
Code Examples
import asyncio
from nexusai_client import AIGateway
async def main():
# 1. Zero configuration auto-failover
async with AIGateway("gemini_free") as client:
res = await client.generate_text("Explain quantum entanglement simply.")
print(f"✅ Served by model [{res.model}] (Provider: {res.provider}):\n{res.text}")
# 2. Live rotation status and cooldown diagnostics
info = await client.get_account_info()
print(f"Active Model: {info.extra_details['current_active_model']}")
print(f"Cooldown Status: {info.extra_details['models_in_cooldown']}")
if __name__ == "__main__":
asyncio.run(main())9. 100% Free Multi-Provider Fallback (auto_fallback_free)
Build zero-cost resilient workflows by cascading across all free providers present in your .env:
import asyncio
from nexusai_client import AIGateway
async def main():
# Cascade: Gemini Free (11 models) -> Groq LPU -> Cerebras CS-3 -> Nvidia NIM -> OrcaRouter -> Mistral -> Cohere -> OpenRouter
async with AIGateway.auto_fallback_free() as client:
res = await client.generate_text("Write a concise summary of AI agents architecture.")
print(f"✅ Served by [{res.provider} / {res.model}]:\n{res.text}")
if __name__ == "__main__":
asyncio.run(main())