To Do:
Sources
https://blog.cloudflare.com/announcing-ai-gateway/
https://www.youtube.com/watch?v=bUe7cGxItvo
Features
How it works
- AI gateway sits between the LLM and the application
- Every time the LLM endpoint is called
- Requests are sent via the gateway
- Tracks when and how the requests are called - time + status(error/ cached/ success)
- It logs request and response too (JSON)
Things that I may have missed
Dev Tool Approach
Product Sense/Design Approach
My Approach
Bugs/ Observations:
Document