On-Device Usage of: gpt4all
gpt4all, a tool for running large language models (LLMs) locally on your own device, is available for macOS, Linux, and Windows.
For on-device usage, instructors at UBC may run this tool.
This tool evaluation is for gpt4all version 3.2.1 available in Sept. 2024.
This tool evaluation is provided for instructors at UBC for their use on their own device. It is to help you make a more informed decision about what generative AI software is available to use if you wish to interact with large language models on your own device, rather than using online third-party tools. Using these tools on your own device can be useful if, for example, you wish to interact with a large language model using personal information or intellectual property – something you are unable to do with 3rd-party online tools, unless that tool has specifically been through a Privacy Impact Assessment.
Without a PIA, instructors cannot require students use the tool or service without providing alternatives that do not require use of student private information
Tool Evaluation
| Tool Name | gpt4all |
| Closed vs. Open Source This refers to whether the tool’s source code is publicly available (open source) or not (closed source). The expected response would be either “Open Source” or “Closed Source”, along with any relevant details such as the repository link and star count if it’s open source. | Open Source. Repository: https://github.com/nomic-ai/gpt4all Star Count: 69,200 |
| Solution Provider This refers to the organization or individual that developed and maintains the tool. The expected response would be the name of the provider and potentially some background information about them. | Nomic — American based start up |
| Regionality of Data This refers to where the data processed by the tool is stored and handled. The expected response would be “Locally Processed” if the data stays on the user’s device, “Cloud Processed” if it’s sent to a server, or a combination of both. | Locally Processed |
| App Analytics & Anonymous sharing of prompts This refers to whether the tool collects usage data and/or allows users to share their prompts anonymously. The expected response would be a description of the tool’s practices in this area. | App analytics and prompt sharing can be opt-in at time of installation by the end user. |
| Prompt Retention & Encryption This refers to how long the tool retains the prompts that users input, and whether those prompts are stored in an encrypted format. The expected response would be a description of the tool’s data retention and encryption policies. | Chats are stored in a binary format, not easily openable, but not necessarily encrypted. Chats can be deleted (along with the history) from the application. Embedded document chunks are stored unencrypted. Embedded documents (and chunks) can be deleted from the application. |
| Administrative Capabilities / Enterprise Offering This refers to whether the tool offers features that are useful for managing its use in a large organization, such as user management, usage reporting, etc. The expected response would be a description of these features, if any. | None |
| Terms of Use This refers to the legal agreement that users must accept to use the tool. The expected response would be a summary of the key points in the agreement, such as the license type and any major restrictions or obligations. | MIT License (Full Use — Limitations on liability) |
| OS Support & Installation Method This refers to which operating systems the tool can run on, and how it’s installed. The expected response would be a list of supported OSes and a description of the installation process | Download from Website/Github — Windows, Mac, Linux. (Requires the end user to enter the click through and terms of use on installation) |
| In App Retrieval Augmented Generation (RAG) Capabilities This refers to whether the tool can use local files to provide more contextually relevant responses. The expected response would be a description of this capability, if present. | Present, can be pointed to file path/folder to ground answers off locally stored files. |
| Third Party Integrations Supported This refers to whether the tool can be connected to other services or platforms. The expected response would be a description of these integrations, if any. | Yes, Can be connected to externally hosted models via API keys. |
Usage Guidelines
gpt4all is software you can run on your PC, Mac, or Linux device. It allows you to interact with downloaded language models without requiring internet access. All of the chats you have with the downloaded models never leave your device. When you first install the software you will be prompted to download one or more language models. After that, you are able to start new (or continue existing) chats with those models.
Additionally, you can point gpt4all at directories of files on your device and then ‘chat’ with them. This functionality you may have seen referred to as Retrieval Augmented Generation (RAG). For example, you may have downloaded several recently published academic papers from arxiv.org and you would like a summary of what is in each of those papers without you needing to read them. gpt4all – via its inbuilt RAG – can help you with that.
Note: gpt4all will store your local chat history on your device. It is recommended you delete your chat history when you are finished with a conversation. Further, gpt4all stores a copy of the data you add to your “LocalDocs” in a non-encrypted database. Once you have finished asking questions of those documents, you are encouraged to delete the documents from gpt4all’s LocalDocs section.