Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Show HN: Skytable's new NoSQL engine with BlueQL, injection safety, better perf (skytable.io)
17 points by sayann on Dec 10, 2023 | hide | past | favorite | 14 comments
Hey HN!

I've been working on Skytable since 2020 and after several iterations from a simple K/V store, we've walked the path to this release. The goal of Skytable is to deliver a solid foundation for building data intensive applications.

Skytable's primary goal is performance and scale. Even with a query language it can outperform K/V stores which use simple commands (benchmarks will be shared in another post).

Several implementations in Skytable (especially around query evaluation and execution) are fundamentally different from SQL and even NoSQL counterparts and there are some entirely new concepts which might make it a little hard to grasp.

BlueQL is a very important part of Skytable and it employs some interesting concepts to try and reduce the surface for injection attacks and tries to be a modern and secure alternative to SQL.

- Source code: https://github.com/skytable/skytable

- Documentation: https://docs.skytable.io

- Rust client driver (more client drivers are underway): https://github.com/skytable/client-rust

I'm here for any questions!

-- Sayan

NB: Clustering and Replication will be released around Q1'24. Choice: AP in CAP. The original post has been updated for clarity.



https://docs.skytable.io/blueql/overview#parameters

Wow, that's very aggressive. It smells to the outside observer like overcorrecting for a real but blown out of proportion problem


I know it might look that way.

But we're just trying to reduce the surface area for mistakes. In the end, one can rest knowing that they didn't at least write vulnerable code, the most common one!


> Skytable's primary goal is performance and scale. Even with a query language it can outperform K/V stores which use simple commands (benchmarks will be shared in another post).

How does Skytable achieves this? In other words, what makes it different than others?


The linked blog post has some notes on the architectural differences. Overall, we use fundamentally different underlying implementations for various components, from the network layer to the storage engine.

If you want a more in-depth review you can read https://docs.skytable.io/architecture

That document will be updated frequently!


I skimmed through the article, still I don’t get it what makes it so fast that it can beat other KV Stores.

Does it use a B Tree / LSM / Bitcask? Or something else entirely?

How does your storage engine compare with sled?

The storage section also mentioned that most data stays in RAM and it is planned to change. Is it still the same? The benchmarks were ran on the in-memory version or disk one (if that is changed)

Also, can you share against which all KV Stores the benchmark was done?

The project looks very exciting and I’d love to know more details


Skytable is currently primarily in-memory (which is to change soon; approximately Q1 next year). The storage engine actually uses various drivers and most of them are AOF-style custom implementations.

Real-time/background compaction and more will be implemented shortly before GA is shipped (this is currently a beta).


Wow, this looks incredible! It looks like the release post has changed significantly since I saw and shared it.

Looking forward to the benchmarks and 1.0!


Thank you. Yes, the early benchmarks definitely seem exciting! I'm working on a comprehensive benchmark with resource usage, latency reports, throughput and more (and benchmarking against more systems now).


Congrats on pursuing this for almost 4 years now. Exciting to see the progress and your continued dedication.


Thank you for following along with us for so long!


How does it measure up against something like MongoDB?


Perf tests will be done soon. The data model is different from mongo and the query language differs as well. Otherwise you should be able to store what you can in Mongo on Skytable as well (excluding full text search, a feature that we're working on).


Any plans on using Jepsen? I searched the issue tracker and didn't see it, but maybe that's not where such a thing would appear in your roadmap


Yes. Jepsen is very much planned once C&R is shipped (ETA Q1'24)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: