Posted by Thomas Ezan, Sr Developer Relation Engineer

Android has supported conventional machine studying fashions for years. Frameworks and SDKs like LiteRT (previously often called TensorFlow Lite), ML Package and MediaPipe enabled builders to simply implement duties like picture classification and object detection.

In recent times, generative AI (gen AI) and huge language fashions (LLMs), have opened up new potentialities for language understanding and textual content era. We have now lowered the limitations for integrating gen AI options into your apps and this weblog publish will offer you the required high-level information to get began.

Earlier than we dive into the specificities of generative AI fashions, let’s take a excessive degree look: how is machine studying (ML) completely different from conventional programming.

Machine studying as a brand new programming paradigm

A key distinction between conventional programming and ML lies in how options are applied.

In conventional programming, builders write specific algorithms that take enter and produce a desired output.

Machine studying takes a unique strategy: builders present a big set of beforehand collected enter knowledge and the corresponding output, and the ML mannequin is skilled to learn to map the enter to the output.

Then, the mannequin is deployed on the Cloud or on-device to course of enter knowledge. This step known as inference.

This paradigm permits builders to deal with issues that have been beforehand troublesome or unattainable to resolve with rule-based programming.

Conventional machine studying vs. generative AI on Android

Conventional ML on Android consists of duties comparable to picture classification that may be applied utilizing mobilenet and LiteRT, or pose estimation that may be simply added to your Android app with the ML Package SDK. These fashions are sometimes skilled on particular datasets and carry out extraordinarily nicely on well-defined, slender duties.

Generative AI introduces the potential to know inputs comparable to textual content, pictures, audio and video and generate human-like responses. This allows purposes like chatbots, language translation, textual content summarization, picture captioning, picture or code era, artistic writing help, and rather more.

Most cutting-edge generative AI fashions just like the Gemini fashions are constructed on the transformer structure. To generate pictures, diffusion fashions are sometimes used.

Understanding massive language fashions

At its core, an LLM is a neural community mannequin skilled on large quantities of textual content knowledge. It learns patterns, grammar, and semantic relationships between phrases and phrases, enabling it to foretell and generate textual content that mimics human language.

As talked about earlier, most up-to-date LLMs use the transformer structure. It breaks down enter into tokens, assigns numerical representations known as “embeddings” (see Key ideas beneath) to those tokens, after which processes these embeddings by a number of layers of the neural community to know the context and that means.

LLMs sometimes undergo two major phases of coaching:

1. Pre-training part: The mannequin is uncovered to huge quantities of textual content from completely different sources to be taught basic language patterns and information.

2. Wonderful-tuning part: The mannequin is skilled on particular duties and datasets to refine its efficiency for specific purposes.

Courses of fashions and their capabilities.

Gen AI fashions are available varied sizes, from smaller fashions like Gemini Nano or Gemma 2 2B, to large fashions like Gemini 1.5 Professional that run on Google Cloud. The dimensions of a mannequin usually correlates with the capabilities and compute energy required to run it.

Fashions are always evolving, with new analysis pushing the boundaries of their capabilities. These fashions are being evaluated on duties like query answering, code era, and inventive writing, demonstrating spectacular outcomes.

As well as some fashions are multimodal which signifies that they’re designed to course of and perceive info from a number of modalities, comparable to pictures, audio, and video, alongside textual content. This permits them to deal with a wider vary of duties, together with picture captioning, visible query answering, audio transcription. A number of Google Generative AI fashions comparable to Gemini 1.5 Flash, Gemini 1.5 Professional, Gemini Nano with Multimodality and PaliGemma are multimodal.

Key ideas

Context Window

Context window refers back to the quantity of tokens (transformed from textual content, picture, audio or video) the mannequin considers when producing a response. For chat use instances, it consists of each the present enter and a historical past of previous interactions. For reference, 100 tokens is the same as about 60-80 English phrases.For reference, Gemini 1.5 Professional at present helps 2M enter tokens. It is sufficient to match the seven Harry Potter books… and extra!

Embeddings

Embeddings are multidimensional numerical representations of tokens that precisely encode their semantic that means and relationships inside a given vector house. Phrases with related meanings are nearer collectively, whereas phrases with reverse meanings are farther aside.

The embedding course of is a key element of an LLM. You possibly can attempt it independently utilizing MediaPipe Textual content Embedder for Android. It may be used to determine relations between phrases and sentences and implement a simplified semantic search immediately on-device.

Prime-Ok, Prime-P and Temperature

Parameters like Prime-Ok, Prime-P and Temperature allow you to regulate the creativity of the mannequin and the randomness of its output.

Prime-Ok filters tokens for output. For instance a Prime-Ok of three retains the three most possible tokens. Growing the Prime-Ok worth will improve the randomness of the mannequin response (study Prime-Ok parameter).

Then, defining the Prime-P worth provides one other step of filtering. Tokens with the very best chances are chosen till their sum equals the Prime-P worth. Decrease Prime-P values lead to much less random responses, and better values lead to extra random responses (study Prime-P parameter).

Lastly, the Temperature defines the randomness to pick the tokens left. Decrease temperatures are good for prompts that require a extra deterministic and fewer open-ended or artistic response, whereas increased temperatures can result in extra numerous or artistic outcomes (study Temperature).

Wonderful-tuning

Iterating over a number of variations of a immediate to attain an optimum response from the mannequin on your use-case isn’t at all times sufficient. The following step is to fine-tune the mannequin by re-training it with knowledge particular to your use-case. You’ll then get hold of a mannequin personalized to your utility.

Extra particularly, Low rank adaptation (LoRA) is a fine-tuning method that makes LLM coaching a lot sooner and extra memory-efficient whereas sustaining the standard of the mannequin outputs.
The method to fine-tune open fashions by way of LoRA is nicely documented. See, for instance, how one can fine-tune Gemini fashions by Google AI Studio with out superior ML experience. It’s also possible to fine-tune Gemma fashions utilizing the KerasNLP library.

The way forward for generative AI on Android

With ongoing analysis and optimization of LLMs for cellular gadgets, we will count on much more revolutionary gen AI enabled options coming to Android quickly. Within the meantime take a look at different AI on Android Highlight Week weblog posts, and go to the Android AI documentation to be taught extra about easy methods to energy your apps with gen AI capabilities!