I had the idea of achieving build isolation by codemodding the Rust implementations of bash and the GNU coreutils (brush+uutils) to use a VFS (wasmtime's cap-std) backed by the real FS and then disabling anything that remained problematic. It kind of works but I'm still not particularly confident in it and I think this is probably the better approach.
The some points of trouble I ran into were dead symlinks left behind on the FS pointing to real files and escape codes interacting with the terminal (e.g. escape codes reading from the clipboard).
> Do you mean isolating build scripts by using a VFS mapped to the real FS and masking away everything the build script should not have access to?
Yes. Essentially the idea was to find the source control root (or something configurable) and mount the real subtree into a VFS where everything interacts with the filesystem through the VFS. I can run wasm compiled versions of apps I don't really trust or run native patched versions that I do.
For background, I'm writing a UI platform in Roc [1] and using just [2] in order to script things. I had extra tokens so I decided to do an LLM port of just over to Roc to exercise the compiler (it's pre-0.1, pushing the compiler leads to crashes) and people won't have to install Rust to write apps. Like wasm, Roc code can't access the outside world without the host providing the access to the outside world so in the process of the port I thought "I don't have to make posix calls, I can put it in sandbox and lie about it" so that's how I got here. I'm fairly close to being able to do hermetic builds so that's a possibility but this is mostly an exploration of whether the idea works or not.
Thanks for sharing details. It does seem like a natural place for a Wasm sandbox + VFS.
Hopefully we can make running these kinds of commands easier as the project progresses.
If you have any interest, please feel to suggest what you want or submit a PR at https://github.com/Automattic/kandelo/. It's easy to get stuck in our own heads working on these tools, and hearing from any real/potential user would be good oxygen for the project.
For an even more niche language, Roc basically became usable in the new syntax about two months ago (still has compiler crashes, there's a good reason it hasn't had a real release) but Opus writes it just fine after a couple corrections to handle the language's quirks.
I noticed this as well and the question caused me to go pretty deep into writing in general.
Switched to a 36 key ergo split with a tweaked alpha-thumb layout (R is on my left thumb, Space is on the right, base layout is Hands Down Vibranium V). Started learning Pitman shorthand (again) then switched to Noory Simplex and stuck with it. Became fascinated with Zaner's Method of cursive where the writing is done entirely through arm movement and using the rhythm of the muscle movement instead of by conscious control over the pen position. At the moment I can write at about 100wpm on a keyboard, 70wpm in shorthand, and 26wpm longhand.
I believe that the improved ability to learn comes from the need to focus on a concept for longer while writing and not on the motion of the writing itself. My ability to retain is better in cursive than in shorthand and I don't notice a significant difference in retention between keyboard and shorthand.
> If any Roc devs are around I'm curious about the use cases for Roc.
It's a general functional programming language that's interested in the constraints and state control properties but not really in the dogma/traditions. As a specific example, it has a for loop statement that doesn't return anything just because sometimes the algorithm is easier to express imperatively. That said, it really is functional, mutating functions/methods require a `!` suffix and `->` (pure) vs `=>` (not) is distinguished in the type system and enforced. The language is fully decidable so type annotations are optional with the arguable exception of the built-in Serde which needs a concrete type to encode/decode. It's also pretty fast, like in the Go range.
I think it has the best error handling of any language in the ~3 dozen I've tried. It's Rust style in general with `Result` renamed to `Try` but the error side of `Try` is an open tag set and can just aggregate so you get the nice parts of the Rust error experience without the downsides. As an example, a coeffect (effectful input) from an example on my server platform:
book! = |req| {
body : { id : I64 }
body = Req.json_body!(req)?
rows = Sql.query!(req.ctx, db_path, "SELECT id, title, author, year FROM books WHERE id = ?", [Integer(body.id)])?
row = Sql.first(rows) ? |_| NotFound("book ${body.id.to_str()} not found")
book = decode_book(row)?
Ok(book)
}
The full set of errors covers malformed utf8, missing/wrong type for id, db errors, the custom NotFound with message, and missing/changed db columns and these plus all the other errors across the app get rolled up and handled in one spot by the error mapping function which rolls the input errors to 400, a 404 for the NotFound, 500s in general in a big match. I have more compact ways to express this in the platform (sqlx) but those don't show off the error handling as nicely.
All in all, it's pretty much just a nice hosted language for doing things.
> Do you see it competing with WASM for the plugin use case?
It's mostly competing with Lua and friends but the host is a platform and not an embedder so the Roc goes on the outside and produces the binary. Roc is particularly well suited for compiling to wasm because all the effects coming from the host is shared. This is actually one of my primary interests in Roc but I haven't really harassed the Roc team about it because they've been busy with the rewrite and wasm module specs have been WIP.
> Why would an app author prefer to expose a Roc layer to their app rather than a WASM layer?
No need for the relatively large WASM runtime would be one of the first ones but Roc isn't really designed to be embedded. I expect to mainly use Roc for app level code on top of Rust for systems level code. I could write app-level Rust but I like functional programming, GC (refcount) is convenient, the error handling is nice, no annotations are nice, super fast compiles are nice, etc.
> Do you see it competing with Gleam for server side http code? Do you see it competing with Elm for client side code?
Sure. As mentioned, I'm experimenting with a server platform that uses pure handlers plus an effect system. I have a RealWorld implementation and in casual benchmarking on my M1 laptop I get 69k req/s for the article endpoint (serialization bound) and 10k going through the article_list endpoint (sqlite bound, 4 table join). The framework also has full and automatic cache invalidation so if I turn on caching I hit 120-140k req/s on both endpoints with no other code changes.
As for GUI stuff, I'm working on a platform (Clay+Solid2) but I don't see any particular reason it wouldn't work.
There isn't much in the way of libraries for Roc at the moment so the split is basically library vs application for my platform. I've been in the Zulip since well before the split but I've been waiting for the new version aside from playing around with the WIP when it first started working for Advent of Code. Reports from the past few weeks indicated it was close enough to ready for me so the web server is my first real effort. It's not generally ready (e.g. the compiler is crashing pretty regularly for me as I push into less common language features) but I'm enjoying myself and I do like the language design.
There is plenty of room for a more interesting nuance once the ecosystem grows:
I'm pretty pleased with my server platform so I'm making a stab at UI with a platform that's Clay and Solid2 ported to Rust. The Solid 2 model has pervasively asynchronous signals so components are written as if they're permanently live and simply don't get run until the constituent signals are ready. My thought process is that this is technically a pure model and only the input changes and effects are impure so there's a pretty clear Roc/Rust split. The platform is still in the assembly process so no actual experience to report. I'll be trying to avoid it but I expect to be doing code generation/compiler hacking in the effort.
On the other side is Luke's roc-signals [1] which explores how the signals model works if all the signal engine code is in Roc with only a minimal backing platform for holding the mutation: "We may not add dataflow analysis passes, dependency-graph extraction, or any new compiler behavior. Everything is ordinary Roc plus a Zig host."
OCaml compiler is incredibly fast. I wonder how it'd fare with Jane Street's extensions for the borrow checker etc in OxCaml, if it's good enough for their HFT I'm sure it's good enough for a new language.
I wrote a toy Scheme implementation in OCaml by using the Camplp4 preprocessor. In benchmarks, it was faster than Gambit Scheme, which compiles through C.
No. I wrote it about 10 years before GitHub existed, and it’s just a toy. All it does is transform core Scheme syntax to Ocaml syntax, converting untyped values in the usual way to a ‘Value’ sum type.
I originally thought it would be slower than the faster Scheme compilers, like Gambit, because of how naive it was, but I was surprised to find that on benchmarks, including compute-heavy ones like fractals and I/O heavy ones like web serving, it outperformed Gambit. That’s really a reflection on Ocaml though, I didn’t do anything special.
If you asked an LLM to do that today, it could probably produce something better for you pretty quickly.
I suspect this "not a systems language" alludes only to OCaml's rather steeper learning curve and until-recently difficulty with multiple threads. I am sure it could roll just fine as a single-threaded compiler language written by a small team, which indeed, it was.
OCaml has often historically been considered a language that's been appropriate to write systems tooling like compilers, runtimes, and unikernels in, even though GC'd languages were/are not often considered for such projects.
I don’t think there’s too many of us on the ‘GC did nothing wrong’ hill.
Reading the average HN opinion, it seems everybody is writing high-performance latency-sensitive systems that would implode if a response would take 1 ms longer than normal.
Sampling bias. Most of the people responding are probably those with a strong opinion because of what they work on. Everyone else is likely relatively indifferent to it.
It is a misconception that GCs only affect latency-sensitive systems. High-performance throughput-optimized systems are also sensitive at ~1µs granularity for different reasons, so GCs are not used there either.
That a GC is adverse to the performance both latency-oriented and throughput-oriented workloads doesn't leave many use cases in "high-performance" systems. Maybe systems that are severely I/O bound but is barely a thing these days.
> Maybe systems that are severely I/O bound but is barely a thing these days.
Any kind of web service is barely a thing today? Which is what 99% of HN posters are working on, hence my comment.
> High-performance throughput-optimized systems are also sensitive at ~1µs granularity for different reasons, so GCs are not used there either
Games are high-performance throughput-optimized systems that have adopted GC languages for 15+ years now, and again a type of application which is much more latency sensitive than most people deal in their day to day.
Nobody is claiming GC is a panacea, but it’s good enough for a lot more use cases people give it credit for.
If you are severely I/O bound it isn't intrinsic, it means your server is badly under-provisioned in the I/O department. Linux on a modern server can push 200 GB/s of I/O. Even if web services were engineered to a standard that could consume that much I/O, which they are not, you would have to be astonishingly wasteful to burn it all.
It is rare to be severely I/O bound because software engineered for I/O performance tends to run out of memory bandwidth first.
Games are not throughput-optimized systems in any conventional sense. They are a canonical example of latency-optimized systems.
I have nothing against GCs, I use them regularly even in performance-sensitive contexts. But too many people understate the adverse impact of GCs on performance contrary to evidence and theory.
I/O bound means waiting on I/O, which isn’t necessarily because it is slow, but because it is simply waiting on data to arrive, like a web service most of its idle time. If your client is dozens of milliseconds away, a GC pause is pretty much invisible, unless you are trying to squeeze every last request/second from a machine (instead of simply scaling horizontally)
That said, from your profile, you seem to work on a very sensitive niche that might colour your opinion, with good reason. What I am claiming is most of us are not building such strict a system.
Even in my toy hobby of OS development a GC isn’t the end of the world unless your goal is to compete with, say, Linux in a some kind of performance challenge, where in that case memory allocation might be the least of your bottlenecks.
It is also a misconception that all GC are born alike, and that don't exit languages with support for value types, stack allocation and GC free memory regions with C like pointer fun if so desired, while being mostly GC enabled.
Could you elaborate on "GCs are not used there [high-performance throughput-optimized systems]"? Are you referring to the cascading effects of tail latency on systems with high fanout?
Sophisticated throughput-optimized systems rely on deep latency-hiding. Schedulers see millions of atomic operations into the future, continuously rewriting the schedule globally to maximize locality and minimize resource contention based on real-time changes to workload, resource availability, and system behaviors.
In short, for each of the millions of in-flight operations (which might only map to a handful of user operations), it is trying to precisely optimize the concurrency, timing, and dependency sequencing such that when operations are executed every resource required is hot, uncontended, and available with high probability. When this works well it dramatically reduces the number of hidden stalls in execution. The schedule is constrained by tail latency requirements; a theoretically throughput-optimal schedule can defer execution indefinitely.
For an analytical database engine, an "atomic operation" is typically a query operation on a database page. A modern server can retire 100M ops/sec. While I am oversimplifying a bit, a 1 millisecond GC pause can blindly wreck the schedule for 100,000 operations in an unpredictable way. In these architectures we try to eliminate all context switches for the same reason which are 100x cheaper.
Practically, 1µs stall is a good heuristic for a noise floor. The schedulers have pretty wide concurrency on big systems, so the implied 100 operations are unlikely to have a dependency. Many stalls that are difficult to precisely control like cache line fills fit in here too.
If there was a GC that had a worst-case stall of 1µs then you could probably use it for these cases. Unfortunately, "low-latency" GCs tend to be more like 1000x that. I don't think there is any way of closing that gap short of putting a GC in hardware.
Real time GC as used by the military in weapons control systems, and on factory automation robots, keeping PTC and Aicas in business, people pay to use them.
TBH, physics limits how latency-sensitive weapons systems need to be and you can largely just disable the GC in these contexts. They use CPUs from the 1990s to do hypersonic terminal guidance. You don’t have to do any performance engineering for many latency-sensitive weapon systems. Could probably write it in Javascript.
For throughput-optimized systems, some of which are real-time, you never see a GC. That loss in performance is simply too large such that the computation becomes intractable. A lot of really poor systems admittedly exist but no one considers them “good”.
I think you may be confused about what "throughput-optimized" means. HFT is not throughput-optimized by definition. The LMAX link literally says it is a latency-optimized system. An optimal throughput-optimized system has unbounded worst-case latency -- the opposite of "latency-optimized".
None of those links contradict anything I wrote, I am already familiar with all of them.
> Most of the people responding are probably those with a strong opinion because of what they work on.
I can assure you that's not the case on here. The people working in truly low latency environments are not commenting on GC threads to begin with because it's a non-starter for them. For whatever reason, there is just a chunk of people that eat a lot of FUD around GCs who are working in the exact domains they thrive in.
The thing that interests me the most is that execution is deterministic. If the inputs to a WASM module are logged you get durable execution and rr style reverse debugging as part of the package.
I ride a Reise & Mueller Birdy Mk3 mostly because I think folding bikes are neat and the Birdy has my favorite fold. The other rationale was that I wanted a nice bike I'd never have to lock up outside in NYC. The intention was to use it as a commuter since not spending on the subway would pay for it pretty quickly but shortly after I got it I started working remote so I've only done commutes on it for a couple weeks of gig jobs and it's mostly a recreational bike.
I went through an extended project to make it faster and wound up with a loop handlebar for body position, replaced the wheelset to move from 355 to 406 for tire selection and did the drivetrain at the same time to accommodate a 9-32 cassette. Between the wheels and the sub-11 tooth sprockets I can pedal up to ~26mph instead of ~20mph on the stock setup (good enough) and the low end is about the same. It doesn't perform like a race bike but it's pretty close to an endurance road bike. I do 20 mile rides a couple times a week on it and I've done a couple centuries.
The Birdy is my main bike but I'm a folding and recumbent enthusiast in general. The addition of the fold or moving the cranks in front of the rider means the obvious solution diamond frame doesn't work and I like seeing the creativity of the solutions. I've also owned a Xootr Swift that I gave away to my nephews, a Bike Friday Sat-R-Day folding recumbent for riding slowly in the parks, and a Baron Optima lowracer recumbent that I prefer for rides over 90 minutes.
It's a shame that it never went into production, but I guess at that time Optima already saw the writing on the wall that two-wheel recumbents wouldn't be profitable for much longer.
I'm particularly delighted that the fold is unique; I've never seen anything similar. Thanks for sharing; this is the sort of thing I look for on the Internet.
I've spent years of my life absolutely alone; gone months without talking to another person. There isn't anything you'll do that will give you the emotional fulfillment of interacting with someone else.
As for living with yourself:
Find some sort of exercise you don't mind doing and make it non-optional. The goal isn't to go all out, just get your heart rate up for half an hour. You won't want to do it sometimes but you still have to go. You can do a crappy job at it and slack off for a day or two but you have to go out. For me this is riding a bike.
Otherwise it's good to be absorbed in something. It's not the same feeling but there is intrinsic satisfaction in learning / building / experiencing things.
I map caps to ctrl and do ctrl-[ to get to normal mode. The main reason is using Vim bindings in other editors where Esc can get intercepted by other bindings but ctrl-[ has always worked everywhere.
My opinion is that going back to normal mode is too important a key to be a key combo, and a weird one at that (is it [ or ] ?). I am pretty sure you can get used to it but we humans get used to anything really, doesn't make it good. My pressing on CapsLock happens at a subconscious level. Quick edit and then punctuate with CapsLock with the pinkie. Some random key combo is not acceptable.
But again my point is that the default sucks. You probably learned a about Ctrl + [ while looking online for alternatives after realizing the default sucked
But you do have to install and configure xcape. By native I meant something that would either be an gui option on your DE or a simple command from something that is already installed on a linux distro like `setxkbmap`
The some points of trouble I ran into were dead symlinks left behind on the FS pointing to real files and escape codes interacting with the terminal (e.g. escape codes reading from the clipboard).