Week 118 - DevOps & CI/CD

The Intern’s Ambitious Leap

Professor Frosty was sipping his hot cocoa in the cozy glow of his workstation when The Intern burst into the room, eyes gleaming with excitement.

“Professor!” he exclaimed, barely able to contain himself, “I’m ready for something bigger. I want to take on a real data challenge—one that involves more than just table tweaks.”

The Professor raised an eyebrow, a grin playing at the corners of his mouth. “Ah, feeling ambitious, are we? Well then, let’s give you something with a bit of a twist.”

Professor Frosty leaned back in his chair, contemplating. “How about a task that will keep Snowflake on its toes and gets a bit of automation into the mix?”

The Intern’s face lit up, and without further ado, Professor Frosty shared the details:

The Mission: Your task is to create a Snowflake table that stays in sync with a YAML file hosted on GitHub. Whenever that file changes, your table will update automatically—think of it as Snowflake magic, powered by a touch of automation.

Your Journey:

1. Set the Scene: Start by creating a Snowflake table called employee_data. This table should hold two columns: first_name and department_id.
2. Populate Your World: Fill in some sample entries to bring employee_data to life.
3. Forge the Link: Create a config.yaml file on GitHub that mirrors the structure of your employee_data table, with the same columns and data.
4. Conjure the Script: Write a Python script that reads config.yaml from GitHub and updates the contents to your Snowflake table. This script will be the bridge between your GitHub-hosted YAML file and Snowflake.
5. Automate the Magic: Set up a GitHub Action to trigger your script every time the YAML file is updated. With this, your Snowflake table will stay in sync with each change—no manual updates needed!

The Intern’s eyes widened as he scribbled down notes. This was exactly the kind of challenge he had hoped for. With a final nod from Professor Frosty, he raced back to his desk, ready to bring this automation to life.

Good luck, Intern! May your code be clean, your YAML well-structured, and your automation flawless!