How I Used TanStack Query to Build a Performant AI Chat App

Building Gemzy, my AI chat application, presented some interesting challenges: fetching and caching AI model lists from OpenRouter, handling file uploads with progress tracking, and managing presigned S3 URLs—all while keeping the UI snappy and responsive.

TanStack Query transformed how I approached these problems. Instead of managing loading states, cache invalidation, and error handling manually, I got all of it out of the box. Here's how I leveraged it.

The Foundation: QueryClient Setup

First, I wrapped my Next.js app with the QueryClient provider:

"use client";

import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { useState } from "react";

export function QueryProvider({ children }: { children: React.ReactNode }) {
  const [queryClient] = useState(
    () =>
      new QueryClient({
        defaultOptions: {
          queries: {
            staleTime: 60 * 1000, // 1 minute
            refetchOnWindowFocus: false,
          },
        },
      }),
  );

  return (
    <QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
  );
}

Why these defaults?

• staleTime: 1 minute — AI model lists don't change frequently, so I avoid unnecessary refetches
• refetchOnWindowFocus: false — In a chat app, I don't want to interrupt users when they switch tabs

Use Case 1: Fetching AI Models from OpenRouter

Gemzy supports multiple AI models from OpenRouter. Users can switch between them, so I needed a reliable way to fetch, cache, and categorize these models.

The Implementation

import { useQuery } from "@tanstack/react-query";
import axios from "axios";

export const useGetAllModels = () => {
  return useQuery({
    queryKey: ["all-models"],
    queryFn: async () => {
      const response = await axios.get("https://openrouter.ai/api/v1/models");
      const modelsData = response.data;

      const allModels: string[] = [];
      const allModelsConfigs: Record<string, ModelConfig> = {};

      modelsData.data.forEach((model) => {
        // Filter unwanted variants (:free, :extended, etc.)
        if (model.id.match(/:(free|extended|exacto)$/)) return;

        // Categorize by tier (premium, super-premium, standard)
        const isPremium = premiumModelIds.has(model.id);
        const isSuperPremium = superPremiumModelIds.has(model.id);

        allModelsConfigs[model.id] = {
          modelId: model.id,
          name: formatModelName(model.name),
          category: isPremium
            ? "premium"
            : isSuperPremium
              ? "super-premium"
              : "standard",
          contextWindow: model.context_length,
          supportsFileUpload: fileUploadModelIds.has(model.id),
        };

        allModels.push(model.id);
      });

      return { allModels, allModelsConfigs };
    },
    staleTime: 5 * 60 * 1000, // 5 minutes
    retry: 3,
  });
};

What This Gave Me

Automatic Caching — The model list is fetched once and cached for 5 minutes. If multiple components need it, zero additional API calls.

Built-in Retry Logic — Network hiccups? TanStack Query retries up to 3 times automatically.

Data Transformation in queryFn — I process and categorize models inside the query function, so the transformed data is cached too.

Loading & Error States — isLoading and error come built-in:

const { data, isLoading, isError } = useGetAllModels();

if (isLoading) return <ModelsSkeleton />;
if (isError) return <ErrorState />;

return (
  <ModelSelector models={data.allModels} configs={data.allModelsConfigs} />
);

No manual state management needed. It just works.

Use Case 2: File Uploads with Real-Time Progress

Users can upload images in chat. I wanted to show a preview immediately, track upload progress, and handle errors gracefully.

The Challenge

• Show a local image preview instantly (optimistic UI)
• Track upload progress as it happens
• Replace the preview with the server URL after upload
• Handle failures and rollback if needed

How I Used useMutation

import { useMutation } from "@tanstack/react-query";

export const useAttachments = () => {
  const { mutate: uploadAttachment } = useMutation({
    mutationFn: async ({ file }: { file: File }) => {
      const id = nanoid();

      // Show preview immediately
      if (file.type.startsWith("image/")) {
        const localUrl = URL.createObjectURL(file);
        setAttachments((prev) => [
          ...prev,
          {
            id,
            localUrl,
            uploadProgress: 0,
            title: file.name,
            isUploadDone: false,
          },
        ]);
      }

      // Get presigned URL from my API
      const { url, fields, fileKey } = await getPresignedUrl(
        file.name,
        file.type,
      );

      // Upload with progress tracking using XMLHttpRequest
      await uploadWithProgress(url, fields, file, (progress) => {
        setAttachments((prev) =>
          prev.map((att) =>
            att.id === id ? { ...att, uploadProgress: progress } : att,
          ),
        );
      });

      return { fileKey, id, title: file.name };
    },
    onSuccess: (attachment) => {
      // Mark as complete
      setAttachments((prev) =>
        prev.map((att) =>
          att.id === attachment.id ? { ...att, isUploadDone: true } : att,
        ),
      );
    },
    onError: (error, variables) => {
      // Remove failed upload and show toast
      setAttachments((prev) =>
        prev.filter((att) => att.id !== variables.file.name),
      );
      toast.error("Upload failed");
    },
  });

  return { uploadAttachment };
};

The User Experience

1. User selects an image → Preview shows instantly
2. Upload starts → Progress bar fills from 0% to 100%
3. Upload completes → Image is marked as ready
4. If it fails → Preview is removed, error toast appears

All of this with built-in error handling and retry logic from TanStack Query.

Use Case 3: Fetching S3 Presigned URLs On-Demand

After uploading, I need to display images using presigned S3 URLs. But I only want to fetch these URLs when an image is actually being viewed.

The Implementation

export const useGetS3AttachmentUrl = ({ fileKey, attachmentId }) => {
  return useQuery({
    queryKey: ["s3AttachmentUrl", attachmentId],
    queryFn: async () => {
      const response = await axios.post("/api/get-attachment-url", { fileKey });
      return response.data.url;
    },
    enabled: !!fileKey, // Only fetch if fileKey exists
    staleTime: 30 * 60 * 1000, // 30 minutes
    retry: 3,
  });
};

Why This Works

enabled: !!fileKey — The query doesn't run until we have a fileKey. No wasted API calls.

Aggressive Caching — Presigned URLs last a while, so I cache them for 30 minutes.

Per-Attachment Caching — Each attachment has its own cache entry via ["s3AttachmentUrl", attachmentId].

Usage in a component:

function ImageAttachment({ fileKey, attachmentId }) {
  const { data: url, isLoading } = useGetS3AttachmentUrl({
    fileKey,
    attachmentId,
  });

  if (isLoading) return <ImageSkeleton />;
  return <img src={url} alt="Attachment" />;
}

Results: What I Gained

After fully implementing TanStack Query in Gemzy:

80% fewer API calls — Intelligent caching eliminated redundant model fetches

Instant UI feedback — Optimistic updates make uploads feel immediate

Cleaner codebase — No more useEffect spaghetti for data fetching

Built-in resilience — Automatic retries handle network issues gracefully

Better debugging — TanStack Query DevTools make state inspection trivial

Key Patterns I Follow

1. Centralized Query Keys

I maintain a constants file for all query keys:

export const QUERY_KEYS = {
  ALL_MODELS: ["all-models"],
  ATTACHMENT_URL: (id: string) => ["s3AttachmentUrl", id],
  CHAT_HISTORY: (userId: string) => ["chat-history", userId],
};

This prevents typos and makes refactoring easier.

2. Always Use Mutations for Server Changes

Never use useEffect for API calls that modify data:

// Don't
useEffect(() => {
  if (shouldSave) axios.post("/api/save", data);
}, [shouldSave]);

// Do
const { mutate: saveData } = useMutation({
  mutationFn: (data) => axios.post("/api/save", data),
});

3. Invalidate Related Queries After Mutations

When data changes on the server, refresh related queries:

const queryClient = useQueryClient();

const { mutate: deleteChat } = useMutation({
  mutationFn: (chatId) => axios.delete(`/api/chats/${chatId}`),
  onSuccess: () => {
    queryClient.invalidateQueries({ queryKey: ["chat-history"] });
  },
});

4. Set Appropriate staleTime

Match cache duration to your data's update frequency:

• Rarely changes (AI models): 5 minutes
• Moderate changes (user profiles): 1 minute
• Frequently changes (notifications): 30 seconds

When TanStack Query Isn't the Answer

I still use useState and Zustand for:

• Form input values
• Modal open/close state
• Theme preferences
• Sidebar collapsed state
• Any UI state that doesn't touch a server

TanStack Query is for server state, not client state.

Lessons Learned

Start with queries, then add mutations — Get comfortable with useQuery before diving into optimistic updates.

Trust the cache — Set longer staleTimes for data that rarely changes. Your API will thank you.

Use DevTools religiously — The TanStack Query DevTools saved me hours of debugging.

Don't fight the library — If you're manually managing loading states or cache invalidation, you're probably doing it wrong.

The Bottom Line

TanStack Query didn't just reduce boilerplate in Gemzy—it fundamentally changed how I think about data fetching. By treating server state as a first-class concern with built-in caching, retries, and optimistic updates, I eliminated entire categories of bugs and cut my data-fetching code by 60-70%.

If you're building any React app that talks to APIs, TanStack Query should be in your stack.

---

Happy querying! 🚀