Generate Nano Blueprint

APK Overview

Generate Nano is packaged within an APK, making it deployable on non-rooted Android devices running the Qualcomm 6490 or 8550 Chipset. The APK includes two main applications: Menu Assistant and Device Aid.

Core Components

Optimized Inference Engine: A GPU-accelerated engine designed to quickly load and run LLMs on Qualcomm devices with minimal overhead.
Fine-Tuned LLMs: Compact language models with built-in guardrails, carefully optimized to perform reliably on resource-constrained hardware.
NPU-Accelerated Voice Module: Combines a voice-specific LLM and OpenAI Whisper-based ASR for fast and accurate voice-to-text conversion using the on-chip NPU.
Advanced Vectorization + Graph Database: A lightweight, high-speed retrieval engine developed in-house to power local semantic search through vector embeddings and graph relationships.
Integrated Fleet Dashboard: Enables real-time system monitoring, log access, and vector database updates via a unified management interface.

Technical Specifications

Model: Interplay Think Quantized Model (4-bit quantized, fine-tuned and optimized for performance)
Speech Recognition: OpenAI Whisper ASR (operates on NPU, supports transcription in 99 languages)
System Requirements: Qualcomm 6490 or 8550 chipset, Android 13 or later

Menu Assistant

Menu Assistant is an application that assists the user (a waiter or customer) with placing orders and interacting with the menu through real-time question and answer. It uses a combination of natural language processing and a locally run LLM to interpret user requests and carry out ordering operations or query the LLM for answers regarding menu items.

Core Features

Natural language processing for ordering and modifying orders
Interaction with locally packaged LLM for menu-related questions
BM25 Search for locating relevant menu items
Support for both local and cloud AI models

Example Queries

User: "Clear cart/Empty cart"

Response: "Your cart has been emptied."

User: "Tell me more about the loaded fries"

Response: "The Loaded Fries are small, crispy fries topped with cheddar cheese, bacon bits, and green onions, served with sour cream for a delicious and flavorful option. They cost $12.00 and are available for small plates."

User: "I want one garlic bread extra crispy and two loaded fries with extra bacon"

Response: "Added 1 Garlic Bread (extra crispy) to your cart. Added 2 Loaded Fries (with extra bacon.) to your cart. Successfully added 2 different items (total quantity: 3)"

Menu Assistant Architecture

graph TD subgraph "Menu Assistant Architecture" INPUT[User Input] NLP[Natural Language Processing] INTENT[Intent Classification] BM25[BM25 Search] LLM[Local LLM] CART[Cart Manager] RESP[Response Generator] OUTPUT[User Output] INPUT --> NLP NLP --> INTENT INTENT -->|Question| BM25 INTENT -->|Order| CART BM25 --> LLM LLM --> RESP CART --> RESP RESP --> OUTPUT end style INPUT fill:#2d2d2d,stroke:#333,stroke-width:1px style NLP fill:#2d2d2d,stroke:#333,stroke-width:1px style INTENT fill:#2d2d2d,stroke:#333,stroke-width:1px style BM25 fill:#2d2d2d,stroke:#333,stroke-width:1px style LLM fill:#2d2d2d,stroke:#333,stroke-width:1px style CART fill:#2d2d2d,stroke:#333,stroke-width:1px style RESP fill:#2d2d2d,stroke:#333,stroke-width:1px style OUTPUT fill:#2d2d2d,stroke:#333,stroke-width:1px

Device Aid

Device Aid is a locally run AI chatbot designed to support retail staff and vendors with setup, training, and troubleshooting while remaining completely offline. By reducing reliance on external support teams, businesses can save millions annually. The assistant helps resolve issues up to 70% faster, freeing staff and support teams to focus on higher-value tasks.

Core Features

Offline AI chatbot for device support and troubleshooting
Retrieval-Augmented Generation (RAG) for accurate responses
Voice interaction through OpenAI Whisper ASR
Document management system for importing, viewing, and deleting documents
Vector Search index that can be rebuilt on demand

Example Queries

User: "How do I turn my DX8000 device on and off?"

Response: "To power on: Press and hold the Power button for about 3 seconds until the Ingenico screen appears. To power off: Press and hold the Power button for more than 3 seconds until the display shows a Power menu, then select Power off. To restart the device, select Restart. To enter Screen saver mode: Press and hold the Power button for 0.5 seconds until the display turns off. Your device will remain on, but asleep. Press and hold the Power button for 0.5 seconds to wake the device and turn the display back on."

User: "How do I set up my Ingenico Desk/5000 card reader?"

Response: "1. Unpack your card reader and accessories. Important: Ensure your card reader is powered off during setup to avoid issues. 2. Connect the PIN pad (if applicable). - Note: The power and internet cables are pre-connected. If you need to disconnect/reconnect, follow these steps: - Power: Connect the power cord from the power adapter to the power port on the Magic Box (lightning bolt symbol). - Ethernet: Connect the Ethernet cable from the switch or other internet access point to the ETH port on the Magic Box."

Device Aid Architecture

graph TD subgraph "Device Aid Architecture" INPUT[User Query] EMBED[Question Embedding] SEARCH[Vector Search] DOCS[Document Retrieval] LLM[Local LLM] OUTPUT[Response] INPUT --> EMBED EMBED --> SEARCH SEARCH --> DOCS DOCS --> LLM INPUT --> LLM LLM --> OUTPUT end style INPUT fill:#2d2d2d,stroke:#333,stroke-width:1px style EMBED fill:#2d2d2d,stroke:#333,stroke-width:1px style SEARCH fill:#2d2d2d,stroke:#333,stroke-width:1px style DOCS fill:#2d2d2d,stroke:#333,stroke-width:1px style LLM fill:#2d2d2d,stroke:#333,stroke-width:1px style OUTPUT fill:#2d2d2d,stroke:#333,stroke-width:1px

APK Installation

Installing the Generate Nano APK on your Android device is a straightforward process. Follow these steps to get started with either the Menu Assistant or Device Aid applications.

How to Install the App on Your Android Device

Download the App:
- On your device, open the appropriate link in your browser to download the APK file:
  - All versions are found here: https://www.iterate.ai/applications/generate-nano
Install the APK:
- Once the download is complete, open your File Manager and locate the APK file (usually in the Downloads folder).
- Tap on the file to begin installation.
Grant Permissions:
- If prompted, allow any necessary permissions (e.g., installing from unknown sources) to proceed with the installation.
Launch the App:
- After installation, the app will appear on your home screen or app drawer.
- Tap the icon to open and begin using the app.

APK Usage Guide

After installing the Generate Nano APK, you'll need to set up the LLM before you can start using the application. Follow these steps to get started:

App Startup & LLM Setup Instructions

Open Settings:
- Tap the red settings icon located in the bottom-right corner of the app's main screen.
Settings Menu Access
Download the LLM:
- In the settings menu, select "Download LLM." The app will begin downloading the LLM (Language Model) directly to your device.
- Do not exit the app until the download is complete and Load Model is available, as this will cause issues with the model initialization.
Download LLM Option

Step 1: Download Model

Step 2: Download Progress
Load the LLM:
- Once the download is complete, tap "Load LLM" to activate the model within the app.
Step 3: Load Model
Start Using the App:
- The LLM is ready after you see the option to "Unload Model" available.
- You can begin running queries and using the chatbot as needed.
- To use CPU simply turn off GPU mode, then Load Model again in order to enable the model.
Adding or Deleting Documents (Device Aid):
- Whenever you delete or add documents to the device, these would have to be Q and A pairs that can easily be used by the LLM.
- After ingesting the documents select "Build Vector Index" to recreate the Vector DB usable by the LLM.

Fleet Dashboard

The Dashboard is an intuitive web-based platform designed to give you full visibility and control over your devices running Generate Nano. With real-time online status, detailed device and app information, and streamlined features to register or remove devices, the dashboard makes managing the deployment simple.

Core Features

System Architecture: The Dashboard is a comprehensive device management platform built with a Flask backend and a modern web frontend.
Device Management: Real-time device registration and status tracking, comprehensive device inventory with detailed specifications, support for multiple device types, device tagging and filtering capabilities.
Real-time Monitoring: Device online/offline status tracking with automatic status updates, device ping mechanism to maintain connection status, last seen timestamps and activity logging.
Log Management: Centralized log collection and viewing interface, log filtering by type (INFO, DEBUG, WARNING, ERROR), full-text search capabilities across log files, log download functionality for offline analysis.
API Integration: RESTful API endpoints for device registration and management, support for device ping updates, bulk operations for device management.