Context-Aware Intelligent Travel Companion – Implementation Overview

This report summarizes the implementation of the Montreal Travel Companion web application and explains how it satisfies the functional, usability, and LLM-related requirements of the course project. The app is a mobile-first React + TypeScript single-page application that uses a Leaflet map, a centralized state store (Zustand), OpenAI’s web-search capable models, and a Firebase Realtime Database–backed login system.

1. Functional Requirements (25 points)

1.1 Detects user’s location and provides location-based recommendations (5 pts)

The application tracks two kinds of location:

• A default context location stored in the global Zustand state (location in ContextState, initialized to downtown Montreal).
• A user-controlled pinned location (set either by tapping on the map or using GPS).

In MapPane, the ClickToPin component listens for map click events via useMapEvents and updates the pinned coordinates; a notification (“Pinned location set”) is also triggered. The bottom-right floating “📍” button in geolocate() uses the browser’s navigator.geolocation.getCurrentPosition API to set the pin to the user’s actual GPS location and pushes a toast with the fixed coordinates.

In the main App component, the “active” context location is always activeLoc = pinned ?? location, i.e., the pin overrides GPS until cleared; this active location is passed into the recommender. The recommendation engine (recommend()) computes distances from this location using a Haversine helper distanceKm() and filters POIs within a configurable radius (3 km by default). The final recommendations are then shown both in the side “Top Picks Nearby” panel and as map markers.

1.2 Detects weather and provides weather-based recommendations (5 pts)

Weather is detected via an LLM-powered helper fetchWeatherFromLLM(), which calls OpenAI’s Responses API with the web_search tool to fetch current conditions for the user’s coordinates. The model is instructed to respond with strict JSON including a short condition string and temperatures in °C and °F. The function parses the JSON, normalizes the condition, infers missing values (e.g., converting °F to °C if needed), and maps it to the internal Weather enum via mapToWeatherEnum().

The App component calls fetchWeatherFromLLM() on mount and then every 30 minutes in a useEffect, updating the global weather and tempC fields in the store. If the call fails, a toast “Weather refresh failed, using last known data” is shown to the user.

The recommendation pipeline enforces weather-aware behavior: outdoor POIs are filtered out in weatherOK() whenever the weather is bad for being outside (rain, snow, or extreme heat). Thus, on a rainy evening, the app will bias toward indoor museums, cinemas, cafés, and restaurants instead of parks or viewpoints. The current weather and temperature are also summarized in the mobile “Dynamic Island” top bar, showing an emoji glyph (☀️, 🌧️, ❄️, etc.) plus the mapped condition and temperature.

1.3 Detects time of day and provides relevant recommendations (5 pts)

Time is stored globally as an ISO timestamp (timeISO) in the Zustand store and is initialized to the current datetime. The “Wizard-of-Oz” controls expose a datetime-local input for the instructor account, allowing simulated time-of-day testing. Because native datetime inputs expect local time strings, a helper toLocalDateTimeInput() converts the stored ISO UTC time into the correct YYYY-MM-DDTHH:mm format, and on change, the value is converted back to a canonical ISO string.

Time directly affects recommendations in two ways:

• The recommender’s isOpen() helper uses dayjs to check the current weekday and current time against each POI’s opening hours table; closed venues are filtered out.
• The LLM prompt context includes dayOfWeek and formatted localTime, which allows the model to understand whether it is breakfast, lunch, dinner, late night, weekday vs. weekend, etc., when proposing additional activities.

1.4 User preferences embedded; recommendations change based on preferences (5 pts)

User preferences are represented by a structured Preferences object in the global store, including:

• indoorOutdoor (indoor / outdoor / either)
• activityTypes (e.g., restaurant, park, museum, café, cinema, viewpoint, shopping)
• cuisines (smoked-meat, pizza, italian, poutine, burgers, asian, coffee, dessert)
• mealTimes (preferred breakfast, lunch, dinner times)

Preferences are stored in and restored from localStorage, so they persist between sessions. The onboarding screen (OnboardPreferences) is shown after login to encourage users to configure their preferences using the shared Controls component; “Skip” and “Done” both set an onboarded flag and advance to the main map.

The Controls panel lets users:

• Toggle indoor vs. outdoor vs. either (with dedicated pills).
• Toggle specific activity types, with indoor-only categories shown when “indoor” is selected, outdoor-only categories when “outdoor” is selected, and both when “either” is selected.
• Adjust meal times using time inputs and select cuisines via toggles, shown only when “restaurant” is part of the selected activities and indoor/either is chosen.

The recommend() function filters POIs by:

• Distance, weather, and indoor/outdoor compatibility
• Selected activityTypes
• Cuisine match for restaurants and cafés
• Opening hours and current time

This ensures that adjusting preferences (e.g., switching from “parks” to “museums”, or from “pizza” to “asian”) immediately changes the set of recommended POIs. Preference changes also trigger a notification “Preferences updated” through the store’s setPrefs method.

1.5 Sends notifications when recommendations are updated (5 pts)

Notifications are implemented centrally in the store as a list of { id, msg } items. The pushNotify() method appends a new item and automatically schedules its removal after a timeout. The main App component renders a “toast stack” in the bottom-right corner, with the newest notification at the bottom; clicking a toast dismisses it.

Recommendation updates trigger notifications in several ways:

• The “✨” button in the top-right of the map calls fetchPOIsFromLLM(). When this completes, the store updates both static POIs and dynamic AI suggestions and pushes the message “Recommendations updated”.
• The free-form ask bar at the bottom uses runFreeformAsk(). After AI suggestions are appended, the UI shows “AI answer ready” as a toast; if the LLM call fails, the app shows “AI search failed”.
• Changing location, time, weather, or preferences also triggers contextual notifications such as “Location updated”, “Time changed”, “Weather changed”, and “Preferences updated”, which indirectly signal that the recommendation logic will refresh with new context.
• Weather auto-refresh failures display “Weather refresh failed, using last known data” to keep users aware of stale conditions.

2. Usability & Design (5 points)

The interface has been designed with Nielsen’s 10 Usability Heuristics in mind. Key mappings are summarized below (0.5 points per heuristic).

2.1 Visibility of system status

• The mobile “Dynamic Island” top bar continuously shows the current coordinates, weather icon, textual condition, and temperature (e.g., “sunny · 21°C”).
• Notifications appear in the bottom-right, confirming actions such as GPS fixes, pin updates, weather refresh failures, AI search results, and preference changes.
• The recommendation panel shows a “Finding great spots for you…” message and an animated progress bar while the LLM is searching.

2.2 Match between system and the real world

• The Leaflet map uses real Montreal coordinates, and POIs correspond to realistic attractions such as “Mount Royal Park”, “Old Port Promenade”, “Notre-Dame Basilica”, and “La Banquise”.
• Category names (restaurant, park, museum, café, cinema, viewpoint, shopping) and weather states (rain, snow, sunny, heat) align with users’ mental models.

2.3 User control and freedom

• Users can clear pins with the “Clear Pin” button in Controls or via the “❌” floating button, returning control to GPS.
• The sidebar (preferences panel) and the recommendation panel can be opened/closed with clear buttons, and the user can log out at any time.
• A “Clear” button at the top of the Preferences card resets filters to defaults.

2.4 Consistency and standards

• Reusable “pill” buttons are used consistently across preferences, weather toggles, and other controls, with a shared visual style defined in index.html.
• Floating action buttons use standard iconography: ⚙️ (settings/sidebar), ✨ (AI recommendations), 📍 (GPS), ❌ (clear pin), 🎤/🎙️ (voice input), and ➤ (send).
• Forms follow familiar email/password patterns with clear labels and error messages in Login.

2.5 Error prevention

• The login form validates email format and enforces a minimum password length before sending any request. It also checks for duplicate accounts in Firebase and rejects bad credentials.
• Leaflet is constrained to Montreal bounds using maxBounds, preventing users from getting “lost” on the map.
• The recommender filters out closed venues and incompatible indoor/outdoor matches, reducing “dead-end” suggestions (e.g., parks in heavy rain or closed restaurants).
• The datetime-local helper prevents invalid date parsing when changing simulated time.

2.6 Recognition rather than recall

• Preferences are expressed via toggleable pills and buttons instead of manual text entry.
• The map and recommendation list show concrete names and badges (e.g., “3.2 km · indoor · food”), enabling users to recognize suitable options visually.

2.7 Flexibility and efficiency of use

• Advanced “Wizard-of-Oz” controls (weather override and simulated time) are only shown to a specific instructor email, keeping the interface simple for normal users while enabling rapid testing.
• Voice input via the Web Speech API allows experienced users to describe what they want (“late-night dessert near Old Port”) instead of typing.
• GPS and pin-clicking both support quick context switching, and the “Clear” preferences button offers a one-tap reset.

2.8 Aesthetic and minimalist design

• The CSS in index.html uses a light, minimal palette, rounded cards, and soft shadows. On phones, the sidebar becomes a slide-in sheet and the map occupies the full viewport.
• The “Dynamic Island”-style bar mimics modern mobile OS status treatments without overwhelming the user.

2.9 Help users recognize, diagnose, and recover from errors

• Login errors (invalid email, short password, wrong credentials, duplicate email) are shown as red text under the form.
• GPS errors and weather/AI failures are surfaced via toasts (“GPS error: …”, “Weather refresh failed…”, “AI search failed”), making failures visible and recoverable.

2.10 Help and documentation

• Inline helper text explains key concepts, such as “Pinned location overrides GPS until cleared”, weather time testing (“Change to test lunch/dinner flows”), and password storage (“We store a SHA-256 hash in Firebase”).
• The onboarding card introduces the purpose of the preferences: “You can change these any time in the sidebar.” :contentReference[oaicite:53]{index=53}

3. LLM Usage (5 points)

3.1 LLM usage inside the application

The project uses OpenAI models in two main runtime features:

(a) LLM-powered POI discovery and recommendations

The module useLLM.tsx defines askLLMJSON() and fetchPOIsFromLLM(). These functions:

• Package context including user location, weather, day of week, local time, indoor/outdoor and activity preferences, user gender, and existing nearby candidate recommendations.
• Call the OpenAI Responses API with tools: [{ type: "web_search" }] and a prompt that forces the model to return strict JSON with summary, an array of items, and optional sources.
• Parse the JSON, convert items with coordinates into POI objects (itemsToPOIs()), and return them along with the summary text.

MapPane uses fetchPOIsFromLLM() in two ways:

• The ✨ button refreshes contextual “Top Picks Nearby” suggestions. New POIs are merged into the global pois set and appended to dynamicPois, and the summary text is shown in the recommendation panel.
• The bottom ask bar (text or voice) performs a freeform query with userPrompt, switches the panel into “AI only” mode, and appends fresh dynamic POIs.

Dynamic POIs are deduplicated by ID so that static and AI-suggested markers do not clash, and the recommendation panel also deduplicates AI results by name.

(b) LLM-powered weather detection

fetchWeatherFromLLM() similarly uses web_search to query a reliable weather provider and returns structured JSON with condition, temperatures, and an optional natural-language summary. This summary can be logged for debugging and is used to validate that the model’s mapping makes sense.

3.2 LLM usage during development

Beyond runtime features, an LLM (ChatGPT) was used extensively as an assistive tool during development in the following ways:

• Architecture design Discussed and iterated on the overall architecture: choosing Zustand for global context (location, time, weather, preferences, notifications, user profile, stage), decomposing the UI into App, Login, OnboardPreferences, Controls, MapPane, and RecPanel, and defining a POI schema with opening hours and tags.
• Recommender logic Used LLM assistance to derive and verify the Haversine distance calculation and to layer filters (distance, weather, indoor/outdoor, activity type, cuisines, opening hours) into a clean functional pipeline that returns Rec objects with human-readable reasons.
• LLM integration & JSON robustness The Responses API returns different shapes depending on model and configuration, which caused initial parsing errors. The LLM was used to inspect sample responses and design the defensive extraction logic that checks multiple possible locations for the JSON text (e.g., data.output[0].content[0].text, choices[0].message.content, etc.). A fallback also strips Markdown code fences when necessary.
• Datetime and timezone bugs The first attempt at using <input type="datetime-local"> simply sliced the ISO string, which led to confusing off-by-timezone errors. With LLM help, a dedicated helper toLocalDateTimeInput() was introduced to convert between ISO UTC and the local datetime format expected by the browser, and to safely handle invalid values.
• Voice input and browser APIs Implementing the Web Speech API required careful handling of vendor prefixes (webkitSpeechRecognition), interim results, and TypeScript typing issues (using any for the recognition instance). ChatGPT helped design the toggleVoice() helper, the ensureRecognition() factory, and error handling logic for cases where the browser does not support speech recognition.
• UI layout & CSS LLM prompts were used to refine the mobile-first layout, including the design of floating FAB positions, the slide-in sidebar, the “Dynamic Island”-style top bar, and the enhanced bottom ask bar with polished styles for the text input and pills.
• Authentication & security The Firebase integration and password handling were designed with LLM support: using SHA-256 hashing in the browser, constructing safe RTDB paths from emails by sanitizing forbidden characters, and using deterministic keys to store user profiles.

3.3 Difficulties and how LLMs helped

Key difficulties and the role of LLMs in overcoming them include:

• Ensuring strict JSON from LLMs: Models sometimes returned explanatory text or Markdown fences instead of pure JSON. The prompt was iteratively refined (e.g., “Return ONLY valid JSON… No extra keys, no commentary”) and fallback parsing logic was added with ChatGPT’s help.
• Handling API response variability: Different OpenAI endpoints and models exposed slightly different JSON shapes, requiring robust extraction. ChatGPT was used to reason about the possible shapes and implement fallbacks that reduce runtime errors.
• Balancing static and AI-generated POIs: The app had to avoid duplicate keys and overlapping list entries when static Montreal POIs and AI-suggested POIs coexisted. ChatGPT suggested deduplication strategies by IDs and names and a combined POI map in App.
• Time and weather edge cases: ChatGPT helped design safe fallbacks (e.g., defaulting to 20°C if no temperature is provided, mapping ambiguous weather descriptions to a small set of enums, and handling missing summary strings) so that the app remains functional even with imperfect external data.

4. Conclusion

The implemented Montreal Travel Companion meets the functional requirements by automatically detecting location, weather, and time; incorporating detailed user preferences; and updating recommendations with clear notifications. A combination of a rule-based recommender and LLM-powered discovery allows the system to provide both reliable, locally constrained suggestions and flexible, web-enhanced ideas. The interface follows the 10 usability heuristics through clear feedback, real-world mapping, strong user control, and a mobile-first, minimalist design. Throughout the project, large language models were used not only at runtime (for weather and POI search) but also as a development partner to refine architecture, fix bugs, and iterate on the user experience.