By 2023, general-purpose conversational AI had become capable and accessible enough that conference app vendors began building chatbot assistants directly into their platforms, positioning them as a way for attendees to get instant answers to logistics questions — session locations, schedule details, venue navigation — without waiting for a staffed help desk or manually searching through the app’s static content.
What the Chatbots Were Actually Good At
The clearest wins for conference chatbots were narrow, factual, high-volume questions that a static FAQ or search function had always struggled to answer well: “what room is the 2pm keynote in,” “when does registration open tomorrow,” “is there a vegetarian option at lunch.” These are exactly the kind of questions that generate a disproportionate share of help-desk traffic at any conference, and a chatbot that could answer them instantly, at any hour, without requiring a staffed desk to be physically present, delivered genuine value by absorbing this volume away from human staff who could then focus on more complex attendee issues.
Chatbots also performed reasonably well at simple personalized queries drawing directly from an attendee’s own registration or schedule data — “what sessions am I registered for,” “where is my next session” — since these queries mapped cleanly onto structured data the app already had, rather than requiring genuine open-ended reasoning.
Where Chatbots Consistently Disappointed
Performance dropped sharply for anything requiring genuine judgment or synthesis rather than simple factual lookup — questions like “which session should I attend if I’m interested in X” required a level of contextual understanding and recommendation quality that general-purpose chatbot implementations in 2023 handled inconsistently, sometimes producing confidently stated but genuinely unhelpful or generic recommendations that didn’t reflect meaningful understanding of the actual program content. The tendency of language models to state uncertain answers with unwarranted confidence is a well-documented trustworthiness concern addressed in guidance like the NIST AI Risk Management Framework.
Attendees also quickly discovered the limits of chatbots when asking about anything not well-represented in the conference’s own structured data — nuanced content questions about a specific session’s actual technical depth, or informal questions about the event’s culture and atmosphere that a human staff member could answer from genuine experience but that a chatbot trained primarily on the event’s logistics data had no real basis for answering well.
The Novelty Effect Faded Fast
Usage data from several conferences that deployed chatbots in 2023 showed a pattern common to many new technology features: high initial engagement driven partly by curiosity and novelty, followed by a meaningful decline in usage once attendees had explored the feature’s capabilities and calibrated their expectations accordingly. Attendees who tried the chatbot for a genuinely complex question, got an unsatisfying answer, and reverted to asking a human staff member instead tended not to return to the chatbot for similar questions later in the same event, having already learned its practical limits.
This pattern suggested that chatbot value was real but narrower than the more ambitious marketing claims around the feature had suggested — a genuinely useful tool for absorbing simple, high-volume factual questions, but not a wholesale replacement for staffed attendee support the way some vendors had initially positioned it.
Staffing Didn’t Actually Decrease as Much as Expected
Organizers who’d hoped chatbot deployment would meaningfully reduce their staffed help-desk requirements generally found the actual staffing reduction more modest than initially projected, precisely because the chatbot’s narrower-than-hoped effective scope meant a meaningful share of attendee questions still required human handling. Organizers who reduced help-desk staffing too aggressively based on optimistic chatbot-capability assumptions sometimes found themselves genuinely understaffed for the complex-question volume that persisted despite the chatbot’s presence.
The more successful deployments treated the chatbot as a volume-reduction tool for a specific, well-defined category of simple questions, staffing help desks accordingly for the remaining, more complex question volume, rather than assuming the chatbot would proportionally replace staffed support across the board.
Cost and Implementation Varied Widely by Vendor
The quality and cost of chatbot implementations varied significantly across event-app vendors during this period, since building a genuinely useful chatbot required both a capable underlying language model and careful integration with an event’s specific structured data — schedule, venue maps, FAQ content — rather than a generic chatbot with no event-specific grounding. Vendors that invested in this integration work produced noticeably better attendee experiences than vendors offering a more generic, loosely integrated chatbot feature primarily to check a feature-parity box against competitors.
Frequently Asked Questions
What kinds of questions did conference chatbots handle well?
Narrow, factual, high-volume logistics questions — session locations, schedule times, dietary options — and simple personalized queries drawing directly from an attendee’s own registration data, like their personal session schedule.
Why did chatbots perform poorly on more complex questions?
Questions requiring genuine judgment or synthesis, like session recommendations based on stated interests, exceeded what general-purpose chatbot implementations could handle reliably in 2023, often producing confidently stated but genuinely unhelpful or generic answers.
Did chatbots meaningfully reduce the need for staffed conference help desks?
Less than many organizers initially expected — since chatbot effectiveness was narrower than optimistic projections assumed, a meaningful share of complex attendee questions still required human handling, and organizers who cut staffing too aggressively based on chatbot capability assumptions sometimes found themselves understaffed.
Why did chatbot usage often decline as an event progressed?
Attendees who tried the chatbot for a complex question, received an unsatisfying answer, and reverted to asking a human staff member typically didn’t return to the chatbot for similar questions later, having already calibrated their expectations about its practical limits.
For related coverage, see our piece on AI session summaries arriving in conference apps.