How can agencies safely use new Chinese-made DeepSeek AI?

The new DeepSeek artificial intelligence model is causing a lot of disruption among AI companies. It was supposedly trained for a fraction of the cost of rival AIs created by U.S. companies and can also calculate its answers using fewer computing resources.

We have been evaluating DeepSeek in the testing lab for a few weeks, pitting it against rivals like Chat GPT-4 in trying to answer a variety of logical and scientific questions. It’s still too early to make any definitive conclusions, but the DeepSeek AI is extremely interesting. 

One of the most noteworthy things about DeepSeek is that it uses a reasoning model where users can watch as the AI thinks out loud. This ability to show its work is something that most AIs can’t do, and the ability to see behind the curtain is not only insightful, but can also help to diagnose problems when the AI gives an incorrect answer. In the lab testing, I’ve even witnessed DeepSeek get ready to give its official answer, only to suddenly recall yet another factor that it needs to consider. It will then sort of debate itself for a while on whether that new factor would or should change its ultimate answer.

For example, being a video game enthusiast, one of the things that I tasked DeepSeek with was creating a version of the classic Tetris game using Python code. The AI spent about 30 seconds debating the rules of the game and which ones should be included before it ever tried to write code. And then, when it was about to generate an answer, it suddenly wondered if it should include player scoring as part of the game’s design. It talked back and forth with itself for a bit, at first arguing that it could program the game without a score mechanic and still perfectly follow our request. But then it suggested that most people keep score when playing games, which is kind of the point of a game, even if I did not specifically request that feature. Ultimately, it added a score keeping function to the game’s code.

My ultimate goal in doing all this testing is to create a guide showing how feds can use DeepSeek safely, and how to make it useful for their agency. It will be very similar to the one I created for federal Chat GPT use, and specifically how feds could make mini GPT assistants to help with their work. However, there are more concerns with using DeepSeek in government compared with Chat GPT.

For one, DeepSeek is a Chinese company, and their privacy policy says that all of the data that anyone enters into DeepSeek is captured by the company, retained forever, and stored on servers within the People’s Republic of China. More concerning, they outright say that they collect more data than they need to answer AI queries, like IP addresses and keystroke patterns. Also, the policy states that the information can be shared with the Chinese government or law enforcement.

Given all that, it’s probably not a good idea for government agencies to enter any data into DeepSeek. However, there is a way that this unique new tool can be used safely.

Installing DeepSeek locally for privacy and security

Even though it might not be safe to use the new AI for government activities through the DeepSeek webpage, it is possible to install it locally on almost any machine running MacOS, Linux or Windows. This can be done in a completely disconnected environment, so long as you are not using the option that allows the AI to search the internet to enhance answers. This was tested in a completely air gapped environment in the test lab and DeepSeek AI performed just fine.

While DeepSeek allows its model to be run locally, they don’t actually provide the means to do so. However, there are several third-party organizations with apps that support the hosting of AI models. The two that we tested are LM Studio and Ollama. The LM Studio app is a little bit more user friendly, but both it and Ollama work great for hosting DeepSeek, as well as many other AIs, in disconnected environments.

In either case, once you download your app of choice, you can then install the DeepSeek AI. You will need to choose which version of the model to use. Larger models with more parameters require beefier computing hardware. The full DeepSeek-R1 model has 671 billion parameters with 37 billion active parameters per token. However, to run that locally would probably require a dedicated workstation with multiple GPUs. It could also run in a cloud instance, but that would negate our attempt to create a fully off-site and disconnected model. The LM Studio app will test your hardware and make recommendations as to which models will run well in that environment.