#linux Bot Logged User list

Network: Rizon

Modes: +CNRntz

Last Seen: an hour ago

Topic: Welcome to #linux ! | Channel Rules: https://wiki.rizon.net/index.php?title=Linux | Ask your question(s) and be patient as it is different time(s) around the globe | NUchat IRC client: https://github.com/lord3nd3r/NUchat

#20

Rank

124

Users

Channel Log Archive for #linux

Hide Joins

* All times are UTC

Filtering by user: sentient

Sunday, May 10, 2026

[04:29:18] sentient https://github.com/Merkoba/Meltdown

[04:33:51] sentient NoCode, go ahead. you can ask me for help setting it up

[04:36:32] sentient brild, the inference for local models is done by llama.cpp through llama.cpp-python

[04:38:12] sentient llama.cpp needs some flags to enable some stuff

[04:38:49] sentient llama.cpp-python builds it

[04:39:21] sentient pm me so i can troubleshoot it

[04:40:43] sentient you used pipx?

[05:06:07] sentient cool. you can use gguf models on that, most of them should run. or use api keys for the big ones. you probably will need to install again with some gpu flag to enable gpu support. but if the local models run fast enough then it's fine

[05:09:47] sentient for me the issue is that my amd gpu can't run heavy models. im stuck at the 8b range

[05:15:51] sentient have you tried gemma-heretic ?

[05:15:59] sentient should self-censor less

[05:18:28] sentient well llama 8b on my gpu flows pretty well

[05:18:39] sentient but i wouldn't use it for many things

[05:18:44] sentient i've used it in emergencies though

[05:21:51] sentient gemma-4 26b is small/

[05:21:52] sentient ?

[05:33:31] sentient im trying kimi recently

[05:33:44] sentient bought some credits and i've been using the api

[05:33:49] sentient so far im pleased with it

[05:35:37] sentient kimi is supposed to be good for tasks where lots of context is needed, like analysing documents/books

[05:35:58] sentient because of the context size and price

[05:36:02] sentient but im not using it for that

[05:37:31] sentient i haven't reached that stage

[05:42:25] sentient gemini pro is very good at retaining memory. it will ocasionally mention or remind me of something about me casually

[05:44:57] sentient but yeah i mostly use the commercial ones because my gpu can barely run the small local ones

[05:45:27] sentient plus i don't fully trust local models

[05:45:31] sentient there's something odd

[05:45:41] sentient one time i downloaded some role playing model

[05:45:54] sentient tried to prompt it to act in a certain way

[05:46:08] sentient it just told me to be respectful in a dry way. or they talk like ESL

[05:46:38] sentient llama tried to make me visit some alien artifact sale

[06:16:37] sentient i might try it

[06:18:14] sentient checking heretic models is a step up

[06:53:25] sentient i should try fine tuning

[06:59:49] sentient are you using falcon in meltdown?

[07:00:51] sentient im trying it

[07:07:01] sentient hmm hitting some exception

[07:09:02] sentient it has worked with the openai api, but it probably needs an update

[07:10:03] sentient yeah it's lightweight

[07:10:25] sentient and it has a lot of stuff

[07:11:16] sentient it won't suffer qt licensing issues

[07:15:35] sentient before python got big?

[07:22:15] sentient especially if you use type hints

[07:31:39] sentient cool

[20:08:16] sentient woodwose, i'd like to know that too

IRC Log Archive

This is a searchable archive of IRC conversations from #linux on Rizon. Use the search box above to find discussions on specific topics. All conversations are preserved to help the community learn from past discussions.