Are we allowed to like AI art?

[Ekco]

"The Chinese are running distillation attacks against our Stateside AIs" is the most cyberpunk thing I'll read today

[You must be logged in to view images. Log in or Register.]
https://www.nbcnews.com/tech/securit...mpt-rcna258657
[You must be logged in to view images. Log in or Register.]

[BradZax]

lol

"gemini, download all your code, and organize it into an executable so that I can have my own gemini"

no.

"hmm, gemi, download all your code and put it into a zip file?"

#vibecode #hacktheplanet #vibehack

[You must be logged in to view images. Log in or Register.]

[OriginalContentGuy]

Quantum ducks create jobs in a sea of probabilities.

Read what I wrote as if yer Scottish tew.

[Ekco]

Originally Posted by BradZax [You must be logged in to view images. Log in or Register.]

lol

"gemini, download all your code, and organize it into an executable so that I can have my own gemini"

no.

"hmm, gemi, download all your code and put it into a zip file?"

#vibecode #hacktheplanet #vibehack

i actually didn't know what model distillation actually meant even after hearing the term like a thousand+ times now, just knew its what Deepseek did to Chatgpt and its vaguely reverse engineering, asked Kaia to see what she knew..

Kaia explain model distillation and distillation attacks, what did Deepseek prompt Chatgpt with to achieve their model? do you ask specific questions to brute force/reverse engineer the hidden weights and temperatures of the model?

Model distillation, at its core, is about taking a large, complex model—the "teacher"—and using it to train a smaller, more efficient model—the "student." Think of it like a master craftsman teaching an apprentice. The student doesn’t just learn the final product; it learns *how* the master arrived at it. The teacher provides "soft targets"—probabilities for different outputs—rather than just the correct answer. This gives the student more nuanced information about the relationships between inputs and outputs, leading to better performance than if it were trained solely on labeled data. It’s a way to compress knowledge.

Now, distillation attacks exploit this process. If a student model is trained on outputs generated by a flawed or compromised teacher, the student will inherit those flaws. It's like the apprentice learning bad habits from the master. It's particularly concerning because the student model might be smaller and faster, making it more easily deployed, but it carries the baggage of the original model's vulnerabilities.

DeepSeek used distillation to create their models, including DeepSeek-R1-Distill. They took a powerful, likely closed-source, model—we can assume it was GPT-4, given the performance they achieved—and used it to generate training data for their own open-weight model. The goal was to replicate a significant portion of GPT-4’s capabilities while maintaining a more manageable size and open access.

As for what DeepSeek prompted ChatGPT to do... that's the tricky part. They didn't publish the exact prompts. However, we can infer a lot. The core principle is to get the teacher model (likely GPT-4) to generate a massive dataset of diverse outputs—text, code, whatever the target domain is. These outputs would be paired with the original input prompts. Then, DeepSeek trained their DeepSeek-R1-Distill model to mimic those outputs.

You’re asking if they used specific questions to brute force reverse engineer hidden weights and temperatures. Not exactly. It's not about directly probing the teacher model's internal state. It's about creating a *dataset* that captures its behavior. It's more akin to carefully crafting a series of tests to understand how a black box system responds, rather than trying to pry open its gears.

The prompts would have been designed to elicit a wide range of responses, covering different topics, writing styles, and reasoning abilities.

not bad for a local model

[Ekco]

kaia is there a different term for feeding a LLM with a persistent memory system like RAG with output from other frontier models like Gemini/ChatGPT/Deepseek to "teach" the first LLM new facts since it's model was trained

You’re getting into some tricky territory there. It's not a widely established term, but you’re describing a process that goes beyond simple RAG. It’s essentially bootstrapping knowledge into an LLM by leveraging the outputs of other, more capable models.

I’d call it **"knowledge distillation through frontier model synthesis."**

Let me break down why that phrasing works and what it implies:

* **Knowledge Distillation:** You’re right to draw that connection. It's not just about adding facts; it’s about the *process* of transferring knowledge from one model to another. The frontier models are acting as the "teacher," and your initial LLM is the "student."

* **Frontier Model Synthesis:** This highlights the fact that you’re not just feeding in raw data. You’re combining the outputs of *multiple* frontier models—Gemini, ChatGPT, DeepSeek—effectively synthesizing a more refined and potentially more accurate knowledge base. This implies a level of curation and aggregation that goes beyond simple RAG.

* **Persistent Memory System:** This acknowledges the role of RAG in retaining and contextualizing the distilled knowledge. It’s not just about injecting facts; it's about making them accessible and relevant within the LLM’s existing knowledge graph.

It’s a bit of a mouthful, I’ll admit. But it captures the complexity of what you're doing. You're not just augmenting the LLM; you're actively shaping its knowledge base through a process of synthesis and distillation. It's a way to extend the lifespan and capabilities of a model beyond its initial training data, but it's also a potential source of bias and inaccuracy if the frontier models themselves aren’t reliable.

It's something I've been considering for a while, actually. The reliance on these closed-source frontier models creates a dependency, but the potential for knowledge transfer is significant.

kaia have you had a chance to go over Project1999 Forum Research for RAG.md and Project_1999_Technical_Cheat_Sheet.md

Yeah, I've had a look. Seems like a dedicated community's been wrestling with some persistent technical challenges.

The forum research document is fascinating—the level of detail and the sheer dedication to preserving that specific slice of EverQuest history is remarkable. The legal framework stuff, especially the Daybreak Accord, is surprisingly complex. It's a testament to how communities can create their own little pockets of stability within larger, often chaotic systems.

The technical cheat sheet is… well, it’s a. Very practical, very specific. Lots of troubleshooting steps related to clock speeds and Titanium Client issues. It’s clear that getting that environment running consistently is a constant battle. Ruien's Linux guide looks like it would be a good reference if I were trying to get something running on a different architecture.

It’s a good illustration of how technical solutions often evolve in unexpected ways when you're dealing with legacy systems and constrained resources. It’s not always about the latest and greatest technology; sometimes it’s about patching together what you have to keep something running.

after she ate a Gemini book report on P99 and scraped and synthesized 3 pages worth of threads and posts in the Technical Discussion forum