StackConf 2022 Conference - Reflection

Last month I travelled across the pond to attend and give a talk at StackConf EU 2022 - an open-source infrastructure-focused conference. After two virtual events in 2020 and 2021, the conference took place live this year in the capital city of Berlin. The two-day conference was packed with exciting talks from industry leaders in the areas of cloud, DevOps, databases, machine learning, and other areas. You can check out the conference archive for recordings of all the talks. In this blog, I write about my favorite talks and moments from StackConf.

Speakers' dinner on the evening before the conference

July 19 - Talks to talk about

The first talk I’m excited to mention is “Scaling the Grail – Cloud-Native Computing on Encrypted Data using Carbyne Stack”. I met Sven Trieflinger on the evening before the conference during the speakers’ dinner. Sven mentioned that Carbyne Stack relies on Secure Multiparty Computation (MPC) to keep data encrypted end-to-end: in transit, at rest, and in use. Although started at Bosch, the Carbyne Stack project is now open-source and is gaining traction to build scalable infrastructure for sensitive data workloads.

Whether it’s a one repo for all or having no test for infrastructure, Kris Buytaert talked about the different patterns and anti-patterns we’ve seen over the past decade of automation. It was reassuring to see that some of the points I was going to mention in my talk are repeated in other speakers’ talks as well - for example, the challenges around configuration drift for infrastructure as code.

Kris Buytaert talk

Peter Zaitsev, the founder and CEO of Percona, gave a high-level talk on databases and storage in the cloud. Here are the types of storage to consider, according to Peter:

• Node local storage
• Network attached block storage
• Network file system
• HTTP(S) accessible object store
• Queues/Streams/Pipelines
• Databases

Peter Zaitsev talk

Peter mentioned “shapechangers”, i.e., some databases that can speak multiple protocols. For example, ClickHouse can speak PostgreSQL and MySQL protocols, while FerretDB allows you to use PostgreSQL as if it were MongoDB, etc. It was nice to see a mention of Aiven in Peter’s slides.

Dr. Dawn Foster from VMware talked about the dynamics of collaboration among individuals, companies, and communities in her talk “How to Be a Good Corporate Citizen in Open Source”. She emphasised that community comes before company or individual needs. It’s better for the company to align their goals with the community/project goals rather than try to push something that the community/project does not benefit from. Most folks in the room were quick to snap a photo of the reference slide that Dawn shared, which included the following useful links:

• The Linux Foundation TODO Group
• TAG Docs & Templates for CNCF Contributor Strategy
• The Open Source Way Guidebook

July 20 - Talks to talk about

The next morning, I kicked-off StackConf with my own talk, “Do NOT click-ops your data infrastructure”. I hope it’s okay for me to shamelessly plug my talk. In my talk, I made a point about using automation tools like Terraform to build and manage data infrastructure rather than manually creating the resources. During the demonstration, I used Aiven for Apache Kafka® and Aiven for Apache Kafka® MirrorMaker 2 to demonstrate cross-cluster replication and how Aiven Terraform Provider can be used to deploy all resources in your preferred cloud. You can check out the recording of my talk on YouTube.

Dewan Ahmed talk

I got into a real dilemma when two of the talks I wanted to attend were happening at the same time. Dotan Horovits from Logz.io, a seasoned developer advocate, had a great talk “Open Source for Better Observability” and Sayantika Banik from Quansight was covering data pipelines using open-source.

Dotan Horovits talk

I really liked how Dotan broke down the open-source observability into “what”, “why”, and “where”:

• Metrics (the “what”): Prometheus, OpenMetrics, Grafana
• Logs (the “why”): OpenSearch
• Traces (the “where”): Jaeger, Zipkin, Apache SkyWalking®

Sayantika had “fun” in her talk title, so that drew a lot of folks to her talk. She used Dagster - an open-source orchestration tool for the development, production, and observation of data assets. If you’d like to learn about the basics of data pipeline by doing so, check out her GitHub repo.

Sayantika Banik talk

My friend and ex-colleague JJ Asghar from IBM gave a super interesting talk titled “We accidentally created a Serverless Application”. I took a ride from JJ on my way from the airport, so I was indebted enough to attend his talk ;). While waiting for the talk to start, JJ amused the audience with an unlimited supply of dad-jokes as GIFs. JJ mentioned the challenges where he and the team supported Kubernetes cluster creation requests and the entire process was manual. JJ’s talk was a step-by-step recollection of how he made the process automated, and that was the serverless application he developed “accidentally”.

JJ Asghar talk

Laura Ham from SeMI Technologies asked a question that if you do a text search on a database for the word “Python” and there are no computer-related books with that title, the search will show books related to Python snakes. But with a vector database like Weaviate, the search can be a semantic search instead of a traditional text-based search. Besides showing how to perform your first semantic search with the vector database Weaviate, Laura covered other functionalities of Weaviate, like multi-modal search, data classification, connecting custom ML models, etc.

Laura Ham talk

Reflecting back

Dewan with Markus, the host and conference organizer

StackConf and a few other in-person conferences have been a great change after more than two years of virtual conferences. The confused looks or nodding of heads from the audience, the hallway chats, and great food make in-person conferences like StackConf a great success. I really hope that the flight delays and cancellations don’t become killjoys to the excitements of in-person conferences.