About as open source as a binary blob without the training data

[email protected]

I'm going to take your point to the extreme.

It's only open source if the camera that took the picture that is used in the stock image that was used to create the texture is open source.
You used a fully mechanical camera and chemical flash powder? Better publish that design patent and include the chemistry of the flash powder!

[email protected]

Well that’s the argument.
Ai condensing ai is what is talked about here, from my understanding deepseek is two parts and they start with known datasets in use, and the two parts bounce ideas against each other and calculates fitness. So degrading recursive results is being directly tackled here. But training sets are tokenized gathered data. The gathering of data sets is a rights issue, but this is not part of the conversation here.
It could be i don’t have a complete concept on what is open source, but from looking into it, all the boxes are checked. The data set is not what is different, it’s just data. Deepseek say its weights are available and open to be changed (https://api-docs.deepseek.com/news/news250120) but the processes that handle that data at unprecedented efficiency us what makes it special

[email protected]

The point of open sourge is access to reproducability the weights are the end products (like a binary blob), you need to supply a way on how the end product is created to be open source.

[email protected]

So i am leaning as much as i can here, so bear with me. But it accepts tokenized data and structures it via a transformer as a json file or sun such. The weights are a binary file that’s separate and is used to, well, modify the tokenized data to generate outcomes. As long as you used a compatible tokenization structure, and weights structure, you could create a new training set. But that can be done with any LLM. You can’t pull the data from this just as you can’t make wheat from dissecting bread. But they provide the tools to set your own data, and the way the LLM handles that data are novel, due to being hamstrung by US sanctions. A “necessity is the mother of invention” and all that. Running comparable ai’s on interior hardware and much smaller budget is what makes this one stand out, not the training data.

Pennomi

Training code created by the community always pops up shortly after release. It has happened for every major model so far. Additionally you have never needed the original training dataset to continue training a model.

[email protected]

So, Ocarina of Time is considered open source now, since it's been decompiled by the community, or what?

Community effort and the ability to build on top of stuff doesn't make anything open source.

Also: initial training data is important.

[email protected]

So its not how it tokenized the data you are looking for, it’s not how the weights are applied you want, and it’s not how it functions to structure the output you want because these are all open… it’s the entirety of the bulk unfiltered data you want. Of which deepseek was provided from other ai projects for initial training, can be changed to fit user needs, and dissent touch on at all how this LLM is different from other LLM’s? This would be as i understand it… like saying that an open source game emulator can’t be open source because Nintendo games are encapsulated? I don’t consider the training data to be the LLM. I consider the system that manipulated that data to be the LLM. Is that where the difference in opinion is?

[email protected]

It's still not open source. No matter hou extendable the weights are.

[email protected]

I mean, this does not help me understand.

Pennomi

Fair enough, it’s not source code, so open source doesn’t apply.

[email protected]

it’s the entirety of the bulk unfiltered data you want

Or more realistically: a description of how you could source the data.

doesnt touch on at all how this LLM is different from other LLM’s?

Correct. Llama isn't open source, either.

like saying that an open source game emulator can’t be open source because Nintendo games are encapsulated

Not at all. It's like claiming an emulator is open source, because it has a plugin system, but you need a closed source build dependency that the developer doesn't disclose to the puplic.

[email protected]

502 Bad Gateway

(slrpnk.net)

[email protected]

A closer analogy would be only providing the binary output of the emulator build and calling it open source. If you can't reproduce building the output from what they provide in what way is it reproducible? The model is the output, the training data and algorithm to build the model based on the training data are the input.

[email protected]

Would it? Not sure how that would be a better analogy. The argument is that it’s nearly all open… but it still does not count because the data set before it’s manipulated by the LLM (in my analogy the data set the emulator is using would be a Nintendo ROM) is not open. A data set that if provided would be so massive, it would render the point of tokenization pointless and be completely unusable by literally ANYONE without multiple data centers redlining for WEEKS. Under that standard of scrutiny not only could there never be an LLM that would qualify, but projects that are considered open source would not be. Thus making the distinction meaningless.

An emulator without a ROM mounted is still an emulator, even if not usable.

[email protected]

Source build dependency… so you don’t have a problem with the LLM at all! You have a problem with the data collection process or the pre-training! So an emulator can’t be open source if the methodology on how the developers discovered how to read Nintendo ROM’s was discovered? Or which games were dissected in order to reverse engineer that info? I don’t consider that a prerequisite to say an emulator is open

So if i say… remove the data set from deepseek what remains would be considered open source by you?

[email protected]

So an emulator can’t be open source if the methodology on how the developers discovered how to read Nintendo ROM’s was discovered?

No. The emulator is open source if it supplies the way on hou to get the binary in the end. I don't know how else to explain it to you: No LLM is open source.

[email protected]

So i still don’t see your issue with deepseek, because just like an emulator, everything is open source, with the exception of the data. The end result is dependent on the ROM put in to it, you can always make your own ROM, if you had the tools, and the end result followed the expected format. And if the ROM was removed the emulator is still the emulator.

So if deep seek removed its data set, would you then consider deepseek open source?

[email protected]

Thanks for the correction and clarification! I just assumed from the open-r1 post that they gave everything aside from the training data.

[email protected]

everything is open source, with the exception of the data

If I distribute a set consisting of emulator and a Rom of a closed source game (without the sourcecode), then the full set is not open source.

So if deep seek removed its data set, would you then consider deepseek open source?

Kind of, but that's like expecting a console without any firmware. The Weights are the important bit of an LLM distribution.

[email protected]

Another theory is that it's the copyright industry at work. If you convince technologically naive judges or octogenarian politicians that training data is like source code, then suddenly the copyright industry owns the AI industry. Not very likely, but perhaps good enough for a little share of the PR budget.