How AppFolio Cut LLM Latency 80–90% With Datadog Observability

Property managers spend a disproportionate share of their working day on tenant correspondence. AppFolio, which provides the software platform used to run properties ranging from single-family rentals to large commercial portfolios, identified this friction point clearly: as many as 50% of a property manager’s working hours were going toward resident communications. For a platform that prides itself on maximizing productivity, that was a gap worth solving with AI.

AppFolio built Realm-X Messages on Amazon Bedrock—an LLM-powered inbox that organizes incoming messages, suggests responses, and flags action items. The technical ambition was clear, but so was the challenge: LLM applications are non-deterministic. Response quality varies. Latency spikes unpredictably. Scaling a generative AI product safely requires visibility that most observability tools were not designed to provide.

The team chose Datadog LLM Observability to monitor Realm-X Messages from the start. The integration with Amazon Bedrock required minimal code annotations, enabling AppFolio to instrument the full LLM chain—from function calls to document retrieval to model calls—without major engineering overhead. Datadog’s dashboards surfaced latency trends and token consumption over time, while out-of-the-box evaluations flagged toxicity and failure-to-answer rates. The cluster map helped the team understand which resident topics the model handled well and which needed improvement.

In the early stages of release, AppFolio spotted a strong correlation between lower latency and user adoption. Using LLM Observability to trace the slowest requests, the team identified specific bottlenecks: inefficient prompts and slow API calls within the LLM chain. Prompt optimization and architecture updates reduced latency by 80 to 90%, which corresponded with adoption climbing nearly 300%. After initial QA setup, AppFolio moved Realm-X Messages into production in under a week.

Today, Realm-X Messages saves property managers an average of five hours per week on resident communications—time redirected to leasing, maintenance coordination, and portfolio growth. The case illustrates a broader pattern emerging in enterprise software: companies building AI-powered features are discovering that the quality of the observability layer determines whether those features scale from pilot to product. AppFolio’s ability to move fast without sacrificing reliability came directly from knowing, in real time, what its LLM was doing.