KernelCI Weekly Newsletter (Week 52)

**Summary**

As the holiday season begins, activity on the KernelCI project has
slowed down. However, we still have some updates to share with you.
This is also the final newsletter of the year, so we would like to take
this opportunity to wish you all a Merry Christmas and a Happy New
Year!

**Milestone Reached: 10GB of Data in Production**

On Friday, December 20th, the KernelCI production environment reached a
significant milestone: 10GB of data stored in its database. This
demonstrates the substantial amount of information being collected and
maintained by the project. It might not sound like much in the era of
terabyte and petabyte storage, but for a project like KernelCI, this is
a notable achievement.

This achievement also marks a critical moment to assess our data
management strategy. The database currently holds approximately 6-7
months of data. As the data grows, we must address how to handle this
expansion effectively to ensure long-term sustainability.

Here are a few considerations:

- **Storage Limitations:**
  With the current production environment running on a Kubernetes (k8s)
cluster comprising three nodes (each with 16GB of RAM and 4 CPU cores),
crossing the 10GB threshold will require additional resources. This
will inevitably increase project costs.

- **Data Retention Strategy:**
  Do we need real-time access to more than six months of data? Or can
older data be archived to a cheaper storage solution? Determining the
balance between accessibility and cost efficiency will be crucial.

- **Archival Solutions:**
  Legacy KernelCI data is already archived. If anyone requires access
to this archived data, please reach out. We aim to ensure that critical
data remains available while managing storage constraints.

**Planning for Efficient Data Storage**

Another important aspect is transitioning from the Legacy KernelCI
production VM, where some data is still stored. This VM is scheduled
for decommissioning soon, prompting the need for a more efficient,
scalable, and sustainable storage solution.

Here are some potential actions:

- **Archiving Older Data:**
  Moving older datasets to long-term, low-cost storage solutions (such
as object storage or dedicated archival systems) to reduce the load on
the production cluster.

- **Data Optimization:**
  Implementing compression techniques, reducing data redundancy, or
adopting more efficient database schemas to minimize storage
requirements.

- **Infrastructure Upgrades:**
  Evaluating the cost-effectiveness of scaling the current cluster
versus adopting a hybrid approach that leverages both on-premise and
cloud-based solutions.

We welcome any input or suggestions regarding these plans. Your
feedback is invaluable as we plan for the future growth of KernelCI.

---

Thank you for being part of the KernelCI community throughout the year.
Let’s continue to work together to improve the project in 2024. Wishing
you a wonderful holiday season and a prosperous New Year!