📄️ Inference Engines
The @optimaxer/web-core library provides various inference engines to generate inferences using small language models (SLMs). This section provides detailed instructions on how to initialize a model and generate inferences using the available inference engines.
📄️ 1. Basic Inference Engine
The @optimaxer/web-core library enables users to generate inferences using various small language models (SLMs) and inference engines. This section provides detailed instructions on how to initialize a model and generate inferences using the BasicLLMInferenceEngine class.
📄️ 2. TransformerJS Inference Engine
The TransformerJS Inference Engine is a powerful tool that allows you to run inference on your models in the browser. It is built on top of the TransformerJS
📄️ 3. Web LLM Inference Engine
The Web LLM Inference Engine is a powerful tool that allows you to run inference on your models in the browser. It is built on top of the WebLLM library
📄️ 4. MediaPipe Inference Engine
The MediaPipe Inference Engine is a powerful tool that allows you to run inference on your models in the browser. It is built on top of the MediaPipe library.
📄️ Client Vector Store
Using the ClientVectorStoreEngine class, users can create vector stores and store them in the end user's browser. There is no need for a central database. Everything is stored in the end user's browser. We also support an embedding engine which runs locally on the end browser. It will create a vector store in the IndexedDB of the browser. This provides a vector search on the browser which is very useful for semantic search and Retrieval-Augmented Generation (RAG) applications (most generative AI solutions need RAG).