@verita84
> I made one too

Link please?

@verita84
> Self hosted AI infrastructure

Intriguing, can you expand on that?

@verita84
> you can run llms for chat, music and image generation

I have many questions. But for now, I want to focus on the technical details of how this works. For example, where does the Trained #MOLE it uses come from? Can it be generated from available source code/ weights/ data, under libre licenses?

@verita84
> Dunno how to answer rest

Let me try asking this way; if I decided to compile all your software from source before installing it, would I be building the Qwen, Ollama and Stable Diffusion models from source? Or would they be downloaded from somewhere as binaries?

@verita84
> The installer and docker setup recommend models to download for chat

Right. So despite being run locally, within Free Code scaffolding, these models are still black boxes, which are themselves proprietary. Even if some (or all) of the source code used to generate the models has been published under a free license, they can't be generated from scratch using that source code. Yes?

(1/2)

@verita84
> You can generate a .gguf however you want and try to run it

I can also pick any cake recipe I want off the web and try to bake it. That doesn't guarantee me an edible cake though ; )

(2/2)

My point is, unless a server project provides a repeatable methods for generating a reliably useful model, running it locally isn't substantially better to running it remotely via an API. The main difference is that the person running the server pays for the compute it uses, instead of the supplier of the model. That seems worse to me, not better.

#AI #MOLE

@verita84
> Unless you have a 32GB GPU , don't even think about doing any development work without an AI subscription. Even 32GB isn't enough

This seems like a good argument for not including generative models in your project. What do you see as the most convincing reasons for including them? Benefits sufficient to outweigh all the downsides we've been discussing?

@verita84
> Image and music generation is fine on lowend gpus

Ok, but what are the *benefits* of generating mediocre images and music by auto-mashing human-created ones? How do they outweigh the costs?

> Qwen 9b is great for bots, summary, and other tasks

What are the benefits of using a Trained #MOLE for these, rather than traditional automation programming, or getting humans to help? How do they outweigh the costs?

'Don't know' is a fair answer, but then, why promote it in your project?

@Jelqberg_Mcjelqstein
@strypey is muted.
and this crap is github-only.

@verita84 @strypey

WRONG. you should NOT be using "AI subscriptions" period, as they are for proprietary models. Paying for proprietary software is even worse than merely using it uncritically, as you clearly do

@jeffcliff
> @strypey is muted

Huh? Why?

@Jelqberg_Mcjelqstein

@Jelqberg_Mcjelqstein
I don't know what this means.

@jeffcliff

@Jelqberg_Mcjelqstein
Imagine I am GenX and I have no idea what this storm of emojis means. Use your *words*. If it just means you think I'm gay, that's factually wrong, but anyone who Mutes on that basis is a mouth-breathing waste of skin, so I don't really give a shit.

But if I'm being accused of something serious, I want to know.

@jeffcliff

@Jelqberg_Mcjelqstein
Ah, I see. You've been at your Mum's brandy again. Get out of the basement and grow up. Muted.