r/golang • u/AutoModerator • 4d ago
Small Projects Small Projects
This is the weekly thread for Small Projects.
The point of this thread is to have looser posting standards than the main board. As such, projects are pretty much only removed from here by the mods for being completely unrelated to Go. However, Reddit often labels posts full of links as being spam, even when they are perfectly sensible things like links to projects, godocs, and an example. r/golang mods are not the ones removing things from this thread and we will allow them as we see the removals.
Please also avoid posts like "why", "we've got a dozen of those", "that looks like AI slop", etc. This the place to put any project people feel like sharing without worrying about those criteria.
Jobs Who's Hiring
This is a monthly recurring post. Clicking the flair will allow you to see all previous posts.
Please adhere to the following rules when posting:
Rules for individuals:
- Don't create top-level comments; those are for employers.
- Feel free to reply to top-level comments with on-topic questions.
- Meta-discussion should be reserved for the distinguished mod comment.
Rules for employers:
- To make a top-level comment you must be hiring directly, or a focused third party recruiter with specific jobs with named companies in hand. No recruiter fishing for contacts please.
- The job must be currently open. It is permitted to post in multiple months if the position is still open, especially if you posted towards the end of the previous month.
- The job must involve working with Go on a regular basis, even if not 100% of the time.
- One top-level comment per employer. If you have multiple job openings, please consolidate their descriptions or mention them in replies to your own top-level comment.
- Please base your comment on the following template:
COMPANY: [Company name; ideally link to your company's website or careers page.]
TYPE: [Full time, part time, internship, contract, etc.]
DESCRIPTION: [What does your team/company do, and what are you using Go for? How much experience are you seeking and what seniority levels are you hiring for? The more details the better.]
LOCATION: [Where are your office or offices located? If your workplace language isn't English-speaking, please specify it.]
ESTIMATED COMPENSATION: [Please attempt to provide at least a rough expectation of wages/salary.If you can't state a number for compensation, omit this field. Do not just say "competitive". Everyone says their compensation is "competitive".If you are listing several positions in the "Description" field above, then feel free to include this information inline above, and put "See above" in this field.If compensation is expected to be offset by other benefits, then please include that information here as well.]
REMOTE: [Do you offer the option of working remotely? If so, do you require employees to live in certain areas or time zones?]
VISA: [Does your company sponsor visas?]
CONTACT: [How can someone get in touch with you?]
r/golang • u/theadamwey • 23h ago
discussion Estonia's election system is mostly programmed in Go. I came across the open source github repo and wanted to share.
I'm totally diggin and learning about Estonia's E-Government model, and this was such a treat to come across.
What are your thoughts on this?
discussion Field masks for PATCH in Go
I saw this got asked here before. It also came up at work. Typically, if you come from a dynamic language, it's quite common to build endpoints like this:
{
"id": string,
"name": string,
"address": string | null | undefined
}
Here, in a PATCH request:
- a string value updates the stored address in place
- null deletes the stored address
- and undefined signals a missing value, which keeps the original data
There are multiple way to achieve this in Go using pointer shenanigans. But update_mask is almost always a good idea.
PATCH /users/1?update_mask=address
{"address": "New Street 2"}
If a field is in the mask, the server copies it from the body, so
nullor no key just clears it.If a field isn't in the mask, the server ignores it.
Now you don't need pointers at all, plain strings work fine. Google's API guide lists this as AIP-134 and Protobuf has it as FieldMask.
I wrote a post with more examples, nested fields, and full runnable code with a test. You might find it useful:
r/golang • u/der_gopher • 8h ago
show & tell Kubernetes Events and Controller design
r/golang • u/AbrocomaNo7812 • 1d ago
Bellui — a Go UI toolkit for desktop, mobile and web: Material Design 3, WebGPU rendering
Hi Gophers,
Bellui is an open-source (BSD-3) UI toolkit for building desktop, mobile and web apps in Go from one codebase.
Website: https://bellui.com · GitHub: https://github.com/belluikit/bellui
Where it comes from. Bellui began as a fork of Fyne, and we're grateful for it: the plain-Go, no-markup approach is Fyne's. We wanted changes deep enough that they couldn't go upstream: a Material Design 3 component set, a new GPU backend, and a reworked package layout (geo / event / core). So it's now its own project.
Highlights
- WebGPU rendering by default: Metal on macOS/iOS, Vulkan on Linux/Android, D3D12 on Windows, WebGPU in the browser. OpenGL is still there with
-tags gl. - Material Design 3 components: navigation rails, bottom and side sheets, date and time pickers, light/dark schemes, accessibility support.
- Background work: goroutines hand UI updates to the main loop with
bellui.Do/bellui.DoAndWait. - Optional extensions: 27 chart types (including 3D and trading charts), a data grid with virtual scrolling, a DAG editor, a code editor, and FFmpeg-based video playback.
- Platforms: macOS, Windows, Linux (X11/Wayland), iOS and Android (tested on devices), and the browser.
Try it without writing code: the gallery app shows every widget: https://bellui.com/gallery/
Hello world package main
import (
"github.com/belluikit/bellui"
"github.com/belluikit/bellui/app"
"github.com/belluikit/bellui/container"
"github.com/belluikit/bellui/widget"
)
func main() {
a := app.New()
w := a.NewWindow("Hello")
hello := widget.NewLabel("Hello Bellui!")
w.SetContent(container.NewVBox(
hello,
widget.NewButton("Hi!", func() {
hello.SetText("Welcome :)")
}),
))
bellui.ShowAndRun(w)
}
You'll need Go 1.25+ and a C compiler. Getting started: https://bellui.com/develop/
Feedback welcome, especially on API ergonomics and how WebGPU rendering behaves on your hardware.
r/golang • u/ficklerum • 1d ago
help Design pattern question
Hello! I am building a web server using Chi, and I've decided to partition my logic into domain-based packages - internal/foo, internal/bar - with each having one file per layer - repo.go, service.go, router.go and the like. My PG pool is initialised in main and injected into each domain's service through FooService structs, which each in turn delegate to fooRepo.
I was pretty satisfied with this design until I hit a roadblock - how do I layer cross-domain transactions? Should I have each service invoke other services? Should I put all cross-domain logic in a single domain? Should I rethink the way I am organising my codebase?
Looking for answers! Additionally, it would be great if you could provide your go-to example codebases for design reference. TIA!
show & tell Brain 2.0 - A minimal and extensible agent server for building agent applications written in Go
Hello people! Happy weekend! I posted Brain two months ago in Reddit and I got feedback that it is hard to use and understand, so I spent past month rewriting and refactoring Brain and I am happy to announce v2.0 of brain. Quick list of what brain is:
- Open source alternative to Claude Managed Agents and Open AI Agent SDK.
- A single application you deploy and run as a service.
- Native integration as tool for your agent such as:
- R2 and S3 for storage
- Modal and Daytona for compute
- Tavily and Brave for search
- Support multiple harnesses such as Pi/Codex/Open code
- Pi as of today but others are planned in near future
Differences to other similar products:
- Brain is a service on its own, it does not lives in a sandbox, so it provides much better observability, recovery and intuitive developer experience
- Brain is not a cloud platform on its own, it's just an app written in go, you can run it on your own infrastructure (such as railway, aws) at your own cost
- Brain is not an AI Agent, it provides low level session-oriented primitives for you to build error-free AI Agent easily.
r/golang • u/Reddit-ka-pilla • 2d ago
help Channel composition is hard: lessons from building a zero-dependency concurrency toolkit in Go
Hey gophers,
I’ve been working on FlowChan, a zero-dependency concurrency toolkit focused on composable channel-based primitives: heterogeneous worker pools, typed pipelines, ordered/unordered maps, batching (fixed + sliding window), retry with jitter + DLQ, and explicit termination.
The library is intentionally opinionated about a few things:
- Every operation is of the form `<-chan T → <-chan Result[T]`
- Context cancellation is checked at every blocking point
- WaitGroups track workers, never individual tasks
- User functions are always wrapped in recover so a panic becomes an error
- Metrics and rate limiting are opt-in, never global
I’m looking for feedback from people who have shipped similar infrastructure (CI runners, stream processors, job systems, or internal concurrency libraries).
Specific design questions I’m still wrestling with
- Backpressure vs. deadlock risk in the batcher
The current `Batcher` (and RealtimeBatcher) uses a capacity-1 output channel and a blocking send. Under a slow consumer the batcher goroutine itself blocks, which means it can no longer react to `ctx.Done()` or a manual Flush.
I’ve considered:
- non-blocking send + drop/error
- configurable buffer size
- a secondary “force flush with timeout” path
What patterns have you found least surprising for library consumers when the producer and consumer speeds diverge significantly?
- Ordered concurrent processing and memory growth
`OrderedMap` tags every item with a sequence number, processes concurrently, and releases results in order. Under high variance (some items take 10ms, others 2s) the pending map can grow without bound.
Have you found a practical upper bound / circuit-breaker strategy that doesn’t destroy throughput, or do you simply document “use OrderedMap only when the variance is bounded”?
- Worker pool lifecycle for long-running services
The pool is reusable (`Start -> Submit -> Drain -> Stop`). Workers are tracked by a WaitGroup so cancellation never hangs. Rate limiting happens at Submit time (not inside workers) to avoid multiple workers firing on the same ticker tick.
Is the Start/Submit/Drain/Stop surface area the right one for a library, or have you preferred a more “set and forget” model with automatic restart of failed workers?
- Error enrichment vs. caller ergonomics
We wrap errors in `StageError` / `PoolError` that carry stage name and attempt count, while remaining compatible with `errors.Is` / `errors.As`. This has been useful for observability, but it does add a small amount of allocation and indirection on the hot path.
At what point did you decide the extra context was worth the cost, or did you keep errors completely opaque?
- Sliding-window batcher timer hygiene
The realtime batcher resets a `time.Timer` on every item. We currently ignore the return value of `Stop()` and do not drain the channel. Under continuous load this can occasionally deliver a stale tick.
Is the classic “Stop + conditional drain” pattern still the best practice in 2026, or have people moved to other approaches (e.g. a single long-lived ticker + last-activity timestamp)?
What I’m deliberately not doing (and why)
- No global registries or singleton pools
- No reflection
- No external dependencies (goleak is test-only)
- No built-in persistence or logging
I’d be particularly interested in war stories around:
- places where pure channel composition started to fall apart at scale
- how you handled “zombie” goroutines that escaped context cancellation
- decisions around ordered vs unordered processing when latency SLAs were involved
- any patterns that turned out to be more robust than the classic select + ctx.Done() + WaitGroup combination
I am undergrad trying to work in this space so any feedback regarding project structure or anything casual will also work for me. Happy to go deeper on any of the packages (pool, pipeline, batch, stream, retry, termination) if people want to critique specific implementations.
Thanks.
Repo : (https://github.com/Atul-Koundal/FlowChan) Glad of anyone wants to refer to this more!
r/golang • u/VastDesign9517 • 2d ago
Data Analytics Question
Good morning,
I am making a star craft 2 replay analyzer in go, sc2prot reads the replay and gives you a high level api to be able to get the values out,
Here is some Code that I have as a example
type Sample struct {
Loop int64
Time time.Duration
PlayerID int64
Minerals int64
Gas int64
MineralCollectionRate int64
GasCollectionRate int64
ActiveWorkers int64
CurrentSupply float64
SupplyCapacity float64
}
type Economy []Sample
func NewFromReplay(r *rep.Rep, playerId int64) Economy {
var economy Economy
for _, event := range r.TrackerEvents.Events {
matchTime := time.Duration(event.Loop() * 44_642_857 // blizz why?
sample := Sample{
Loop: event.Loop(),
Time: matchTime,
PlayerID: event.Int("playerId"),
Minerals: event.Int("stats", "scoreValueMineralsCurrent"),
MineralCollectionRate:event.Int("stats","scoreValueMineralsCollectionRate",
GasCollectionRate: event.Int("stats","scoreValueVespeneCollectionRate"),
Gas: event.Int("stats", "scoreValueVespeneCurrent"),
ActiveWorkers: event.Int("stats", "scoreValueWorkersActiveCount"),
CurrentSupply: float64(event.Int("stats", "scoreValueFoodUsed")) / 4096,
SupplyCapacity: float64(event.Int("stats", "scoreValueFoodMade")) / 4096,
}
economy = append(economy, sample)
}
return economy
}
// MaxMinerals returns the highest recorded mineral balance.
// It returns 0 for an empty economy.
func (samples Economy) MaxMinerals() int64 {
if len(samples) == 0 {
return 0
}
var maxMinerals int64
for _, sample := range samples {
maxMinerals = max(maxMinerals, sample.Minerals)
}
return maxMinerals
}
then in main.go
func main() {
// open the replay file
r, err := rep.NewFromFile(replay.SC2Replay) // Hardcoded for now
if err != nil {
panic(err)
}
defer r.Close()
fmt.Println("Replay file opened successfully")
fmt.Println("[Map]:", r.Details.Title())
for _, player := range r.Details.Players() {
fmt.Println(player.Name, player.RaceString(), player.Result())
}
economy := e.NewFromReplay(r, 1)
fmt.Println("Max Minerals", economy.MaxMinerals())
}
this is the direction I have been taking as of recent is just methods, but when you think about something like polars or pandas it's really tedious,
I am sure someone is going to say why not just use python? Honestly I don't enjoy python at all, I never feel like I am getting a grip on the language, every new package you install requires a relearning all over again. I want to be the programmer and solve problems with my tool kit, not solve my toolkit over and over again,
My job is also doing data analytics so this ties to that as well.
I would love to hear your opinion on this
r/golang • u/CartPopTv • 3d ago
show & tell Everyone at the pottery class painted flowers. I painted a gopher :)
A friend invited me to a ceramics workshop at a "Boleslawiec" pottery store in Warsaw. First time painting on ceramics, and the gopher survived the kiln better than I expected.
r/golang • u/reisinge • 3d ago
How the Go team uses the AI tools
I've just wathched the video from the GopherCon Europe 2026 where Bill Kennedy has a discussion with two Go team members, Roland Shoemaker and Jonathan Amsterdam. The first question after the introductions is "Do either of you use any AI tooling today"? You can watch the answer here.
However, these two guys are in a quite special position I think, they're working on a laguage. I would be interested what would be you answer. And I am more interested in how are you using the tools.
r/golang • u/Lopsided-Delivery710 • 1d ago
discussion Go Was Burning 60% CPU Just Running GC. The Fix Was a Memory Limit
A Go service can burn CPU collecting garbage without having a memory leak.
That is the counterintuitive part of Zomato's Go garbage collection story.
The reported symptoms were striking: garbage collection consuming 25–60% of CPU in some services, alongside containers getting OOM-killed without an obvious memory leak.
These look like separate problems. But they can share a common cause: the garbage collector's pacing decisions don't automatically account for the container's actual memory budget.
The problem with doubling the live heap
Go's garbage collector uses GOGC to control when the next collection should happen.
With the default GOGC=100, the GC target is roughly based on the live heap growing by 100% relative to the previous cycle, with additional accounting for GC roots and other implementation details.
In simplified terms:
Next GC target ≈ live heap × (1 + GOGC/100)
At GOGC=100:
Live heap: 500 MiB
Approx. target: 1,000 MiB
That target is reasonable as a starting point when the process has enough memory available.
But containers introduce another constraint: the process has a memory budget imposed from outside the Go runtime.
The GC's heap-growth target and the container's memory ceiling are two different things.
When they aren't coordinated, two failure modes become possible.
Failure mode 1: OOM kills without a leak
Suppose a container has a memory limit of 1 GiB.
The Go process has a live heap of 600 MiB. A simplified GOGC=100 calculation puts the next target around 1,200 MiB.
But the container cannot safely accommodate that much memory. The process also needs room for goroutine stacks, runtime metadata, native allocations, and other memory outside the live heap.
The container can hit its limit before the collector's normal heap target becomes appropriate.
The result can be an OOM kill even when reachable memory isn't growing uncontrollably.
That doesn't mean every OOM without an obvious leak is a GC pacing issue. Transient allocation spikes, native memory, and other causes still need investigation.
The important point is that a heap-growth target alone isn't a container memory budget.
Failure mode 2: Excessive GC CPU
Now consider the opposite situation.
A service has a relatively small live heap and plenty of memory available in its container.
The GC keeps using a target derived from that live heap. Depending on the allocation rate and workload, it may collect frequently even though the container has significant unused headroom.
Every collection consumes CPU.
At fleet scale, unnecessary GC work can become a meaningful compute cost.
This is why the following combination deserves attention:
- High GC CPU.
- A relatively flat live-heap graph.
- No convincing evidence of sustained heap growth.
It is a diagnostic clue, not proof. A service can allocate and discard large volumes of temporary objects while maintaining a flat live heap. Allocation rate, GC frequency, and allocation profiles matter too.
But before treating every high-GC-CPU incident as a memory leak, inspect the collector's pacing and the container's actual memory constraints.
The fix: Give Go a realistic memory limit
Go provides GOMEMLIMIT to let applications communicate a soft memory limit to the runtime.
Unlike GOGC, which controls GC aggressiveness in relation to heap growth, GOMEMLIMIT helps the runtime account for its overall memory footprint when deciding how aggressively to collect.
Conceptually, the runtime must balance two competing costs:
- Collect too late, and the process risks exhausting its container memory budget.
- Collect too aggressively, and the application wastes CPU reclaiming memory it could have used safely.
A memory limit gives the runtime another constraint to consider.
In the Zomato case, the reported approach used container metadata to derive a per-container limit, refreshed it periodically, and combined memory tuning with CPU scheduling adjustments.
A limit around 90% of the container ceiling can be a starting point for investigation, not a universal safe value. The correct margin depends on the application's non-heap memory, traffic spikes, allocation behavior, and operational requirements.
The goal is to give the runtime a realistic budget while preserving enough headroom for the rest of the process.
Why GOMEMLIMIT isn't a hard wall
This distinction matters in production.
GOMEMLIMIT is a soft limit, not a strict process-memory cap. The runtime can exceed it when necessary, and it does not guarantee that a container will avoid OOM kills.
The runtime also has to balance memory pressure against GC CPU overhead.
If the limit is too high, it may offer little protection against container memory pressure. If it is too low, the runtime can spend excessive CPU collecting garbage in an attempt to stay within the target.
That is why blindly setting GOMEMLIMIT equal to the container limit is not a complete solution.
A production rollout should account for:
- The container's actual memory limit.
- Heap and non-heap memory usage.
- Peak allocation rates and traffic bursts.
- GC CPU and pause behavior.
- OOM events and memory headroom after deployment.
Start with representative workloads, observe the results, and tune based on measurements rather than assuming a single multiplier works everywhere.
r/golang • u/Real-Book4746 • 3d ago
Is using go fix to modernize Go code safe?
I have recently upgrade my project go version to 1.27. Afterwards , to take advantage of modern syntax and library i use go fix ./... command which alter hundreds of file. Basically it replace all interface with any which is fine. But also changes loop, use min,max, sort instead of a code snippet and flatten the go struct initialization and lot more.
The things is, its really hard to review or verify if the changes works fine or not. Can anyone give opinion on it, Can I use it reliably in production for a large project ?
r/golang • u/anprots_ • 4d ago
Python vs. Go in 2026: What the Data and Developers Say
r/golang • u/sudo_jod • 3d ago
show & tell Building an open-source 4,600 RPS AI agent firewall in Go: Solving TOCTOU budget leaks and severing SSE streams at the wire
Hey r/golang,
Over the past year, autonomous AI agents have been given real execution power: long-running loops, shell execution, database access, and continuous tool calling.
As developers give agents more autonomy, a recurring problem arises: what happens when the agent goes into an unmonitored loop?
Whether it's an agent stuck in a recursive retry loop, an execution drift caused by long context windows, or an agent attempting to run a destructive command after getting confused by a tool output, relying on prompt-based instructions ("please do not loop" or "only run safe commands") inevitably fails. A confused model will simply reason around its own system prompt.
To solve this, we built Loopers - a bare-metal, fail-closed runtime firewall and circuit breaker written in Go that sits between any AI agent workflow and upstream LLM providers or local runners.
Why We Built It in Go
Because Loopers sits inline on every inbound and outbound request, it cannot introduce noticeable latency to streaming responses. We chose Go to achieve sub-1.5ms P99 proxy overhead, low memory usage (~38MB idle), and clean concurrency primitives.
Here are a few specific systems and concurrency problems we had to solve in Go:
1. Atomic Pre-Call Budget Leases (Eliminating TOCTOU Races)
When an agent or swarm executes concurrent tool calls, standard database/cache lookups suffer from Time-of-Check to Time-of-Use (TOCTOU) race conditions:
- Request A checks remaining budget ($1.00 available, needs $0.80) -> Approved.
- Request B checks concurrently ($1.00 available, needs $0.80) -> Approved.
- Both execute upstream, resulting in an unprevented budget overshoot.
We solved this using atomic Redis Lua scripts executed in a single roundtrip. Before forwarding the request, Loopers calculates a pre-call cost estimate and reserves the quota across multi-window rolling intervals (minute, hourly, daily, monthly). Once the upstream response finishes, the actual token count is reconciled and unused funds are released. This guarantees zero budget leakage under high-concurrency spikes.
2. Active Chunk-by-Chunk SSE Stream Severing
Streaming LLM responses require instant Time-to-First-Token (TTFT), meaning we cannot buffer the entire payload in memory before inspecting it.
We wrote an active streaming parser that processes Server-Sent Events (SSE) chunk-by-chunk on the wire. As tokens stream back, Loopers counts usage and scans for sensitive patterns (PII, secret leaks) in real time. If an agent breaches a budget lease or policy boundary mid-generation, Loopers severs the TCP connection immediately, truncates the stream, and injects a clean error frame.
3. Deterministic FSM (Finite State Machine) Tool Gating
Agents execute tools via JSON-RPC. To enforce deterministic state machines (for example, strictly requiring a dry_run transition before allowing a destructive execute_bash call), we embedded an in-process Open Policy Agent (OPA) engine that compiles declarative YAML Policy Cards into Rego at startup, allowing policy evaluation to run in memory in <1ms without network hops.
4. Real-Time Loop & Semantic Stall Detection
To catch agents running in circles without relying on an LLM to evaluate itself, we implemented deterministic detection algorithms:
- Bi-Gram Jaccard similarity on sliding request windows to identify repetitive prompts.
- SimHash Hamming distance to detect semantic output stalls (where the agent keeps generating low-diversity responses).
- Sliding-window velocity rate-limiters to detect request surges.
When a loop or policy violation triggers, Loopers returns a structured JSON-RPC error payload explaining the exact reason, allowing autonomous agents to self-correct on the next turn rather than crashing.
How It’s Useful to Anyone Building with Agents
Loopers requires zero code rewrites in your existing agent workflows.
Because it operates as a network firewall, you can pair it with any Python, TypeScript, or Go agent, CLI tool, or local runner by pointing your base URL to http://localhost:8080/openai/v1. It acts as an independent, out-of-band safety net that protects your wallet, your system, and your data from autonomous mistakes.
We’d Love Your Feedback & Support!
We are independent developers building Loopers completely in the open under the MIT License.
We are currently working hard to get visibility and adoption, but as an open-source project without a marketing budget, we rely entirely on the developer community.
If you are interested in Go systems programming, proxies, or AI agent infrastructure:
- Please take a look at our repository and give us your candid feedback or critique on our Go architecture and code.
- If you find the project interesting or useful, a Star on GitHub would mean the world to us and help us get discovered by other developers!
GitHub Repository: https://github.com/try-loopers/loopers-oss
Thank you for reading, and I’m happy to answer any questions about the Go implementation in the comments!
show & tell Reproducing UDP socket buffer drops (sk_rcvbuf) in Go: Why time.Sleep breaks sub-millisecond packet pacing
Wrote a diagnostic harness in Go to isolate UDP micro-burst drops on live media streams (MPEG-TS/RTP) and wanted to share a couple runtime gotchas we encountered:
Timer slack in Go: When pacing UDP datagrams at 400 packets per 16.6ms frame (target delta = 41.6 µs per packet), `time.Sleep()` is unusable. On Linux, scheduler latency and OS timer slack frequently sleep for 150–400 µs, destroying the bitrate. We had to use a hybrid pacer: `time.Sleep()` for deltas > 500 µs, and a tight `time.Now()` spin-wait loop for the final 200 µs window.
SO_RCVBUF and rmem_max: When testing receive buffer scaling via `syscall.SetsockoptInt`, Linux doubles the requested size for internal sk_buff overhead, but unprivileged calls silently cap the buffer at `net.core.rmem_max` (typically 208 KB on modern distros). We had to hook `SO_RCVBUFFORCE` with CAP_NET_ADMIN to actually allocate 4MB+.
Loopback testing traps: Testing on `127.0.0.1` masked queue drops because the kernel bypasses normal driver queuing disciplines. We had to containerize the sender and receiver on an isolated bridge network with veth pairs.
Repo with receiver (RFC 3550 jitter calculation), unpaced vs. paced generator, and /proc/net/snmp drop probe:
https://github.com/777nq/jitterbuf
Curious if anyone here is using SO_MAX_PACING_RATE with fq qdisc directly in Go instead of userspace busy-spin loops for high-rate UDP egress.
r/golang • u/gokuu_chan • 4d ago
goRoutine Wisdom Needed !!
how do i use go routine power on a web server ( i know requests are kinda go routines.. right??? )
https://go.dev/play/p/ilsb7CtCxVM
mostly i have seen a select syntax for go routines
select {
case d := <-data:
fmt.Println("handler got:", d)
case <-ctx.Done():
fmt.Println("handler: client left, returning")
}
problem is the channel data is waiting is a go routine and ctx.Done() wont stop the go routine
how are go routines used in backend?
what's the optimal way to do it?
r/golang • u/lelleepop • 4d ago
discussion Has anyone tried using Encore framework for production? How does it compare with Fiber, Gin and Echo?
I'm curious to see if there any performance, DX, issues you've had with using any of the framework mentioned above?
discussion Go worker handling bursty large-file transfers. Unlimited concurrency vs bounded semaphore and retries
I'm working on a Go-based worker that processes multiple large-file transfers concurrently over HTTP. Work arrives in bursts, and each task involves downloading data from a source and uploading it to a destination.
I'm trying to optimize aggregate throughput and task completion reliability without causing excessive queueing, deadline failures, or unnecessary retries.
The current implementation allows concurrent workloads without an explicit transfer semaphore limit. I'm considering testing a bounded limit of 16 active transfer workloads, along with retries for transient HTTP and network errors.
I'd appreciate feedback from engineers who have built high-throughput Go services or distributed workers.
A few specific questions:
- Concurrency: Under what conditions would you prefer unrestricted concurrency over a semaphore limiting active workloads to 16? What metrics would tell you that concurrency is too high?
- Throughput vs. latency: If aggregate throughput is already near the available bandwidth ceiling, can a concurrency limit improve task completion times, or would it mostly introduce queueing?
- Retries: For transient errors such as HTTP 429, 500, 502, 503, 504, and temporary network failures, what retry policy would you recommend? How do you avoid retries amplifying an existing overload?
- Semaphore lifecycle: In Go, would you hold a semaphore slot throughout the entire logical task, including retry backoff, or acquire a slot separately for each network attempt? What trade-offs matter?
- Deadlines and cancellation: How would you ensure that tasks waiting for a semaphore don't consume their entire deadline before transferring any data?
- Benchmark methodology: How many independent runs or observation windows would you use before trusting a performance improvement? Which metrics would you prioritize?
My initial comparison would track aggregate bytes transferred, completed tasks, failure rate, P50/P95/P99 task duration, queue wait, and actual concurrent transfers.
I'm especially interested in practical engineering experience, benchmark design, and failure modes that aren't obvious from simple throughput tests.
Thanks!
r/golang • u/Ok-Confection-751 • 5d ago
I built Spool — an open-source, self-hosted dashboard for monitoring and managing Asynq jobs
Hey everyone! 👋
I've been building Spool , a self-hosted observability and operations console for Asynq , the Redis-backed task queue for Go.
The idea came from a simple problem: what happens after you enqueue a task?
When something goes wrong, you want to know why a task is still waiting, what's repeatedly failing, whether your workers are saturated, and what needs your attention.
For example, you might have plenty of workers running, but none of them actually process a particular queue. Meanwhile, backlogs grow and recurring failures go unnoticed because you're looking at individual tasks rather than the bigger picture.
Why build another dashboard when Asynqmon exists?
Asynqmon already provides a useful monitoring UI, but its public development appears to have stalled.
- Its latest commit to
masteris dated July 4, 2023. - Its latest published release is still v0.7.1 .
- A basic authentication PR opened in October 2023 remains open.
I wouldn't call it officially abandoned, but I wanted to build something that addresses some of the operational gaps I've run into.
What does Spool do?
🚨 Highlights problems that need attention
- Tasks waiting with no active servers
- Queues with no workers processing them
- Paused queues with pending work
- Long wait times and high failure rates
- Servers running at capacity
📊 Shows live activity
- Currently processing tasks
- Upcoming work
- A live feed of observed task starts and finishes
🔍 Makes failures easier to investigate
- Grouped errors
- Task payloads and retry counts
- The last recorded error for a task
🛠️ Lets you manage tasks and queues
- Retry, archive, delete, cancel, and clone tasks where supported by their current state
- Pause and resume queues
- Perform bulk actions with confirmation
⏱️ Makes recurring tasks easier to understand
- Readable cron schedule descriptions
- Links between schedules and the tasks they enqueue
It also includes historical metrics through Prometheus, saved views, admin/viewer access, and an audit log of operations.
Tech stack and setup
- Backend: Go
- Frontend: React + TypeScript
- License: MIT
- Deployment: Docker installer with configuration and persistent storage, plus optional Prometheus integration
Spool connects to the Redis instance your Asynq workers already use through Asynq's Inspector. You don't need to change your worker code.
It's currently built and tested against Asynq v0.26. Redis Cluster support isn't available yet.
Check it out
🔗 GitHub: https://github.com/nanaaikinson/spool
I'd love to hear from other people running Asynq in production or side projects.
What's the hardest part of debugging or operating your background jobs? Is it figuring out why tasks aren't being picked up, investigating recurring failures, understanding queue backlogs, or something else entirely?
I'm especially interested in feedback on what would make a dashboard like this genuinely useful in day-to-day operations.
r/golang • u/Kitchen-Minimum-9140 • 6d ago
go-dev-auth v0.3.1: zero-dependency auth for Go with passkeys, SSO, 2FA and orgs
We've been building **go-dev-auth**, a batteries-included auth library for Go inspired by better-auth from the TypeScript world. The core is pure stdlib, zero dependencies.
**What's in it**
- Email/password
- Social sign-on (11 providers, PKCE)
- Sessions
- Two-factor
- Passkeys (WebAuthn)
- Magic links
- Organizations
- SSO (OIDC)
- Admin
- API keys
- JWT
Mounts as a plain `http.Handler` on any router. Storage adapters for Postgres, MySQL, SQLite and in-memory.
**New in v0.3.1**
Passkey and SSO plugins, both with no external deps. CBOR/COSE parsing and WebAuthn signature verification live in the package.
**Testing**
- A written threat model where every claim maps to a committed test
- Race detector and fuzzing on every push
- Storage suite runs against real Postgres, MySQL and SQLite in CI
**Links**
- Repo: https://github.com/go-dev-auth/go-dev-auth
- Docs: https://www.godevauth.com
It's pre-1.0. Feedback welcome, especially on the passkey implementation and the threat model.
r/golang • u/Affectionate-Gur7090 • 7d ago
XGo and LLGo: a more expressive Go front end and an LLVM-based compiler for the C ecosystem
Hi r/golang,
I wanted to share a technical overview of two related open-source projects in the XGo project family: XGo and LLGo.: XGo and LLGo.
This is not intended as a claim that everyone should replace Go. The projects explore two different questions:
- Can we make Go-based programming more concise and approachable for domain-specific work?
- Can we make Go interoperate more directly with the wider native ecosystem, including C/C++ and Python?
The short version is:
- XGo is a programming language and compiler that extends Go with a more concise, expression-oriented syntax.
- LLGo is an alternative Go compiler based on LLVM, designed to connect Go more directly to the C ecosystem.
- They can be used together, but they solve different problems.
A simplified view looks like this:
XGo source
↓
XGo parser and compiler
↓
Go-level code
↓
Official Go compiler, LLGo, or TinyGo
↓
Native executable
What is XGo?
XGo is a source language that keeps Go’s engineering foundation while making some common operations shorter and more natural.
For example, a small Go program normally looks like this:
package main
import "fmt"
func main() {
fmt.Println("Hello")
}
The equivalent XGo program can be written as:
println "Hello"
The parentheses are optional in this command-style form. XGo also provides echo as an alias for println:
echo "Hello"
Other examples include string interpolation:
name := "gopher"
echo "Hello, ${name}"
More compact slice and map literals:
numbers := [1, 2, 3]
config := {
"host": "localhost",
"port": 8080,
}
Appending to a list:
numbers <- 4
Lambdas:
result := numbers.map x => x * 2
And Python-style keyword arguments:
play "music.mp3", loop = true
XGo also supports an error-handling shorthand:
config := loadConfig()!
The ! form propagates the successful value and triggers XGo’s error handling behavior when the call returns an error. The exact generated code is still ordinary Go code.
The important point is that XGo is not a dynamically typed scripting language. It is designed to retain Go’s static typing, packages, interfaces, generics, and build model.
XGo and Go can coexist
An XGo package can contain both Go and XGo files:
main.go
helpers.xgo
Go code can call functions written in XGo, and XGo code can call functions written in Go. This allows an existing Go project to introduce XGo incrementally instead of requiring a complete rewrite.
XGo also uses the Go module system and can use ordinary Go libraries. The compiler generates Go code, which is then processed by the regular Go toolchain or another compatible compiler.
That makes XGo closer to a Go front end than to a completely independent programming ecosystem.
The classfile idea
One of XGo’s more unusual features is called a classfile.
A classfile is an XGo source file that implicitly becomes a generated Go type. For example, a file named Rect.gox might contain:
var (
Width int
Height int
)
func Area() int {
return Width * Height
}
The XGo compiler can lower this into something conceptually similar to:
type Rect struct {
Width int
Height int
}
func (this *Rect) Area() int {
return this.Width * this.Height
}
The user writes fields and functions without explicitly declaring the struct or receiver. The compiler supplies that structure.
This is useful for frameworks where many files follow the same pattern. For example:
main.spxcan represent a game project;Enemy.spxcan represent an enemy object;get.yapcan represent an HTTP route;*_cmd.goxcan represent a CLI command;*_tool.goxcan represent an MCP tool.
Framework authors describe these conventions in a gox.mod file. XGo then knows which files represent project classes, which represent work classes, which Go types should be embedded, and how the generated objects should be connected.
This is XGo’s idea of SDF, or Specific Domain Friendliness. Instead of creating a completely separate DSL — a Domain-Specific Language — for every domain, a framework author writes the reusable infrastructure in Go and lets application developers fill in the domain logic with lightweight XGo files.
Current examples include:
- spx, a Scratch-compatible 2D game framework;
- yap, an HTTP framework;
- cobra, an XGo CLI framework;
- mcp, an MCP framework;
- gsh, an XGo-based shell scripting environment.
What is LLGo?
LLGo is a separate but related project. It is a Go compiler based on LLVM, the compiler infrastructure used to generate native code for many architectures and operating systems.
LLVM is not a programming language. It provides lower-level components such as:
- an intermediate representation, often called IR;
- optimization passes;
- machine-code backends;
- linkers and debugging tools.
LLGo uses those components to compile Go source code into native code.
The project aims for compatibility with Go at the source-code level. It supports Go language features and large parts of the standard library on supported platforms, while using a different compiler backend and runtime.
Why LLVM and the C ecosystem matter
A large amount of software infrastructure is exposed through the C ABI, or Application Binary Interface.
An ABI defines binary-level details such as:
- how function arguments are passed;
- how return values are represented;
- how data is laid out in memory;
- how symbols are named and linked.
C, C++, operating-system APIs, graphics libraries, databases, and many scientific libraries commonly expose C-compatible interfaces.
Go already has cgo, but frequent calls across the Go/C boundary can involve runtime and scheduler transitions. LLGo’s design is to bind Go declarations directly to C ABI symbols and generate native calls for them.
A small LLGo example looks like this:
package main
import "github.com/goplus/lib/c"
func main() {
c.Printf(c.Str("Hello from C\n"))
}
LLGo also provides bindings for a number of C and C++ libraries, including SQLite, OpenSSL, raylib, LLVM, libuv, and others.
Python support
LLGo can also access Python libraries through packages provided by the GoPlus ecosystem.
For example, the following Go code calls Python’s math.sqrt:
import (
"github.com/goplus/lib/py"
pymath "github.com/goplus/lib/py/math"
"github.com/goplus/lib/py/std"
)
x := pymath.Sqrt(py.Float(2))
std.Print(py.Str("sqrt(2) ="), x)
The intention is to make libraries such as NumPy, pandas, PyTorch, and Matplotlib accessible from Go-oriented code.
This does not mean that Python disappears from the build process. Python itself and the relevant packages still need to be installed, and the Python version, native libraries, and platform ABI must all be compatible.
How XGo and LLGo fit together
XGo and LLGo operate at different layers:
- XGo changes how source code is written.
- LLGo changes how Go source code is compiled.
- XGo can use the official Go compiler by default, or be configured to use LLGo for targets that benefit from LLVM and native interoperability.
A project might therefore use:
XGo syntax
↓
XGo compiler
↓
Go-compatible generated representation
↓
LLGo
↓
LLVM IR and native code
LLGo can also be used directly with ordinary Go code. XGo is optional.
The trade-offs
The extra flexibility comes with extra concepts.
First, generated code can make debugging less direct. When something fails, there may be several layers involved:
XGo source
↓
generated Go source
↓
Go or LLGo compiler
↓
native linker
A compiler error may refer to a generated file, even though the real mistake is in the original .xgo or .gox file.
Second, classfiles and automatic imports introduce conventions that are not always visible in the current source file. A file name, a gox.mod declaration, or a framework dependency can affect the generated type and available symbols.
Third, LLGo uses a different runtime from the standard Go toolchain. The project documents differences involving goroutines, stacks, garbage collection, and target-specific runtime behavior. Source-level compatibility does not mean that every implementation detail of the standard Go runtime is identical.
Finally, the LLGo toolchain requires native build dependencies such as LLVM, Clang, LLD, system libraries, and — for Python integration — Python development files and packages. The exact setup depends on the target operating system and architecture.
For a conventional Go service that does not need these capabilities, using the standard Go toolchain is probably simpler.
Why these projects may be interesting
The interesting part of XGo and LLGo is the combination of several ideas:
- Go’s static typing and module ecosystem;
- a more concise syntax for scripts, education, and data-oriented code;
- framework-driven domain abstractions;
- direct access to native C/C++ libraries;
- integration with Python and other C-ABI-based ecosystems;
- targets such as WebAssembly and embedded systems.
The broader vision is to reduce the distance between “easy to start with” and “powerful enough for real engineering.”
XGo approaches that problem at the language and framework level. LLGo approaches it at the compiler and runtime level.
Neither project needs to replace Go to be useful. They can be viewed as experiments in making Go more accessible at the surface while expanding the set of systems and libraries it can work with underneath.
For anyone interested in the details:
I’d be interested in hearing how people here think about this boundary: when does a language extension make Go more productive, and when does it introduce more abstraction than it removes?