Category Archives: SQL Server

This T-SQL Tuesday is hosted by the one and only James Serra – literally my go-to guy for data lakes and BI architecture. I am so tickled to be writing this right when am reading his book on ‘Deciphering Data Architectures: Choosing Between a Modern Data Warehouse, Data Fabric, Data Lakehouse, and Data Mesh’ (small book and easy to get through, recommended). James’s call to us is to blog on career risks we’ve taken.

I am a risk taker.

I strongly believe in doing work that stimulates and energizes you, rather than treating work as just a job. Some people regard this belief as elitist. I don’t.

I come from a poor country, and I have seen firsthand the toll that ill‑suited work takes on people—jobs taken purely to pay the bills, regardless of health, temperament, or personality. We spend eight to ten hours a day at work on average. If we dislike what we do, it is bound to have detrimental effects over time. Wanting work that helps you be creative and does not slowly wear you down is not elitism; it is self‑preservation.

My career reflects that belief.

I began as a COBOL programmer, transitioned to VB, then became a SQL Server DBA. I spent many years as a DBA before moving into a Database Engineering role, and I am now a Data Engineer. I am also close to graduating with a master’s degree in data science, and my next goal is to transition into an AI Ethics role.

Along the way, I took several significant risks.

1. Stepping outside data entirely

Seven years into my DBA role at a healthcare company, I realized the job had stopped helping me grow—financially and intellectually. I left for a cloud architect role at a startup, which turned out not to be what I had hoped. When I tried to move on, I couldn’t find another role that met my needs.

At my former employer, the infrastructure and Windows support team had an open sysadmin position. I had limited sysadmin experience, but I believed I could learn, and the team consisted of people I knew and trusted. I interviewed and got the role.

I worked as a sysadmin for two years. I can’t say I loved it, but I learned a lot —about monitoring, alerting, patching, managing heterogeneous server environments, and, most importantly, teamwork. When I returned to data work, that experience stayed with me and continues to serve me well.

2. Leaving the comfort of deep expertise

My next major risk was moving from a DBA role—where I had significant experience—into a Database Engineering role. I had always leaned toward application DBA work due to my programming background, but I was also experiencing serious burnout from on‑call rotations and constant operational stress.

I wanted a role with no on call, one that allowed me to focus more on building and programming. I found such a role, but it required relocating to another town. I took the risk and moved.

That Database Engineering role turned out to be the best career decision I’ve made. I enjoyed the work immensely and learned more than I could have imagined.

3. Returning to school

Another major risk was pursuing a master’s degree in data science. I have a background in statistics and have always enjoyed working with R and solving analytical problems on the side. Graduate school had long appealed to me, but it was either too expensive or required in‑person attendance—neither of which was feasible.

Post‑COVID, many universities began offering affordable, fully online programs. One of them was the University of Massachusetts. The program fit my budget, and the curriculum aligned perfectly with what I wanted to learn. I enrolled, and I am enjoying the experience.

4. Embracing Databricks and analytics

The most recent risk I took was moving from a Database Engineering role into a Data Engineer role. My organization decided to adopt Databricks as its data lake platform. While I had some exposure to BI in previous roles, I was far from an expert.

The analytics teams were heavily invested in Databricks, and learning it felt like a natural complement to my data science studies. I’m now three months into working with Databricks and loving every bit of it. I hope this path will eventually take me into a data scientist or AI Ethics role—and I’m looking forward to the ride.

Thank you James, for hosting.

The last T-SQL Tuesday of the year is hosted by my good friend Mike Walsh. Mike’s call for us is to end the year on a poignant note – write a note each to your past me and future me, on things you wish you knew, or learning plans or anything you’d like to say.

My notes are as below –

Note to My Past Self (10 Years Ago)

I wish you had realized that admiring others shouldn’t make you lose sight of your own worth and potential.

I wish you knew that real friendships take time to grow.

I wish you understood that if all you share with someone are conversations about community politics and tech, that relationship probably won’t last.

I wish you expected people to ask about you too—not just the other way around.

I wish you learned to let go and move on easily—whether from people or jobs that have outlived their value.

I wish you knew how to pick your battles and not let every little thing upset you.

But despite all this, you did your best. Life is a learning experience, after all.

Note to My Future Self

I’m glad you’ve learned to find joy in doing things for their own sake—not for rewards, even in tech.

I’m glad you kept learning what you wanted to learn, despite the challenges.

I’m glad you figured out who your true friends are—and held on to them.

I’m glad you learned to trust life and let go of what doesn’t matter.

And I’m glad you still find time to smell the roses and be grateful.

As we close out the year, I’m reminded that growth is a journey—one that stretches across decades, shaped by lessons learned and hopes carried forward. Here’s to continuing that journey with curiosity, resilience, and gratitude. Thank you Mike, for hosting and wishing everyone a happy and safe holiday season.

I was privileged to host yet another T-SQL Tuesday, for the month of August, 2024. My topic was on Managing database code. I was worried about getting responses, given that most of the community is no longer on X/Twitter and my invite didn’t seem to get much attention. I was wrong. People seem to keep track of this by other means and the response was excellent. Summary as below.

Kevin Chant (b/l) talks of his experience and preferences around repos and CI/CD pipelines using Azure Devops/Github for  SQL Server, SQL Pools and Microsoft Fabric Data Warehouses. He talks about using Projects for doing state based migration and migration based deployments, where repo structure can be quite different from a standard one. His post has a lot of links to technologies he has used and includes an invite to engage with him, for those who want to learn more.

Long time Data Platform MVP, MCM and friend from down under – Rob Farley (b|l) differentiates between ‘where the database code lives’ and ‘where do you keep your database code’ . From a consultant’s perspective, I can totally understand the difference between those two questions. It was revealing to me, that as a full-time employee for six years, that difference did not even occur to me because we inherently own what we manage. He talks of doing whatever fits in with the client’s existing processes – while recommending a script-based repository if they don’t have one. He also seems to use the method as I do personally – Powershell based SMO, to script out objects.

Well known conference speaker/DEI Champion and Data Platform MVP Deepthi Goguri (b|l) talks of a simple version control process using RedGate source control and GIT, that she uses where she works.

Tom (b) talks of 3 approaches to version scripts – a state-based approach, a migration-based approach and a hybrid approach. They talk of the common aversion to consider database code as repo worthy as well.

Data Platform MVP, Conference speaker and WIT champion Deborah Melkin (b|t) writes about many types of scripts and their importance to workflow for database developers. She emphasizes the importance of having code in source control, regardless of how one chooses to do it. She also has a session on ‘Database deployments demystified’ for this. I found the recording here and recommend it.

Andy Levy (b|l) blogs about usual Visual Studio database projects along with powershell to generate scripts and keep up the SQL Server based repo he manages. He also speaks of managing a MySQL repo with just scripts needed for changes, and having large swaths of code that is not in the repo.

Data Platform MVP Eitan Blumin(b|l) blogs about the advantages of GIT over TFS/VSS, and his favorite tool for CI/CD, Azure Devops. He also explains the advantages of SSDT over other tools in detail.

Data Platform MVP, speaker and author Richard Swinbank(b|l) talks of basic principles behind code management (resilience, repeatability), and the process he uses to manage code.

Jeff Mlaker(b) talks of the pitfalls of not using source control for database code, with some interesting personal stories including one on merging different versions of code when no source control existed.

Lastly, long time Data Platform MVP, DevOps Advocate and currently owner of the T-SQL Tuesday blog series, Steve Jones(b|l), talks of the importance of getting code into repo and the tools he uses to accomplish the same.

My own humble two cents on this topic is here.

This is my T-SQL Tuesday #5. It is a great way to understand multiple perspectives and experiences around a topic, get more blog traffic and gain more contacts to add to your community. Contact Steve if you wish to host one.

Thank you to everyone who responded here. Your responses enriched my knowledge of this subject and helped me find more community. #sqlfamily #sqlwinners.

This is my own contribution to the T-SQL Tuesday I am hosting – on managing database code.

I am from the older generation – where the farthest we went with managing code was use VSS/TFS to get the latest version and deploy it. In some cases, I’ve managed it. If we wanted to search for existence of an object or its usage, we simply searched the database – using some custom tools/dmvs or 3rd party tools like RedGate’s SQL Search. History of who changed what – well that came from their tickets or if it was code – from code comment history. With the advent of GIT, all that changed. I did not adapt to GIT very quickly. I went, kicking and screaming. I hated the clunky interface. I hated not knowing what the command actually did, but still had to use it. I did not see the point of learning what I thought to be over engineered version control. But over time, I’ve learned its value. I can find almost any code that has been committed. I can see who changed what, and when, very easily. I can search for code usage and objects, really quickly. Granted, the interface is still clunky and merge conflicts aren’t fun when they happen, but the advantages override all that, significantly. There is one place where we can easily see all the code related to a database. We can evaluate how complex it is and even use it for legal reasons as code base of the company.

As of now, I use GIT for source control extensively. At work all our database objects are scripted out and have their repo. When we make a change, the new objects are auto scripted with the changes.

I am a fan of SMO, for this purpose and have scripts I’ve developed myself. Granted, SMO only works for SQL Server. I am learning more on similar tools for Postgres. I am often frustrated that Redshift has none. I strongly advocate using GIT and putting all your database objects into a repo using whatever means you can get. (Or develop your own means). It is worth a lot more than you think.

I am excited to host the T-SQL Tuesday blog party for August 2024. I’ve done this many times, but I always remember when I was new to the community and how much it helped me gain recognition. Thanks to Adam Machanic for starting this initiative and to Steve Jones for keeping it going.

This month’s topic is ‘Managing Database Code’. I initially called it ‘How do you manage your database repo?’, but Steve pointed out that many people in the database community might not know what a ‘repo’ is. So, we simplified it to ‘Managing Database Code’. Managing code has been a big part of my work as a database engineer for the past six years.

A repo, for those who are not aware, is just short for ‘repository‘, a place to store stuff. In this context, a place where ‘keep’ code. It is version control for code. I remember the days before repos were common. We used tools like VSS/TFS and searched through tickets and comments to track changes. We used DMV-based queries to find objects. Now, with repos, everything is much easier. I can’t imagine working without one.

I’m curious about how others manage their database code. Here are some questions to consider:

1. Where do you keep your database code? Is it in a GIT-based repo, or just in the database the old-fashioned way?
2. If it is in a repo, how is it created? Do you script out individual scripts and add them, or do you use tools to script out the entire database, like SSMS scripter, SMO-based scripts, or third-party tools? This could apply to any relational database, not just SQL Server.
3. How do you keep your repo up to date when code changes? Depending on what your shop does, you may or may not have a well evolved CI/CD pipeline. If you do, explain how it works. If you don’t, that’s fine – we just want to know where the code resides, if it is outside the database.
4. If you don’t have a repo, why not? How do you manage things like searching code or finding who made changes and when?

I look forward to reading your responses. The rules

• Publish your post on Tuesday 13th of August , time zone does not matter.
• Include the T-SQL Tuesday logo at the top of your post and link it back to this invitation post.
  • A comment letting me know you have a contribution is ideal but if that’s not doable, just send me an email!)
• If you’re on X/Twitter, share your post using #tsql2sday.

If you want to know more, check out the T-SQL Tuesday Rules of Engagement.

Thank you!

I recently graduated with a Master’s in Strategic Communication from the University of Delaware. Attending the graduation ceremony with my sister made it a truly memorable experience.

Here’s a bit about why I chose this degree and what’s next for me.

During the Covid crisis, I sought a new focus amid the trauma and confusion of losing four dear family members, including my Dad. Initially, I looked into Data Science programs, but many were beyond my budget or required prerequisites I couldn’t meet. I held a three-year bachelor’s degree and a three year masters (MBA) from India, from over 30 years ago. Degree evaluation is possible but course by course transcript evaluation is rather difficult. Also most programs needed a 4 year bachelors, not 3.

Online master’s programs were also very limited at the time, so I opted for this degree as a stepping stone towards future studies in Data science and analytics. I appreciated the curriculum and the school, especially the courses on sales, PR, and sales data analytics, which I formerly knew very little of and found fascinating. I also made good friends among my classmates and faculty. Graduating with an A has positioned me well for pursuing further studies more aligned with my professional interests.

Now, I am applying to three Data Analytics programs. While some might find it unusual to jump into another demanding master’s program so soon, I see it differently. I love learning in an academic setting and believe it will enhance my career prospects significantly, if I make the right choices on the degree and learn with it.

A big thanks to friends in the community who supported and inspired me on this journey – Blythe Morrow, Randolph West, Kevin Kline, Joe Fleming, Cindy Gross and Daniel Maenle. I am grateful for your support and your friendship.

Onwards and upwards!

Thanks for reading.

My dear friend Josephine Bush a.k.a HelloSQLKitty hosts this month’s T-SQL Tuesday. Josephine’s call to us is to share our understanding of the multitude of job titles available out there and what they mean to us.

While many job titles exist, each role’s essence is heavily shaped by the operational dynamics of the business and how responsibilities are allocated among individuals. A rough and high-level comprehension of these titles can be outlined as follows:

• Database Administrator (DBA): Responsible for High Availability/Disaster Recovery (HA/DR), Backups/Restores, security measures, including audits, storage management, code promotions, and sometimes even code reviews. Encompasses Continuous Integration/Continuous Deployment (CI/CD) and automation, and may also be referred to as a ‘Database Reliability Engineer’.
• Database Engineer (DBE): Tasked with managing database code, architecture, and data process flow. Engages in Extract, Transform, Load (ETL) processes, pipelines, and assumes responsibility for data integrity and health.
• Database Architect: Often involves a blend of DBA and DBE roles, with a substantial infusion of cloud architecture expertise.
• Data Warehouse Engineer/BI Architect: Primarily concerned with reporting, data warehousing, and data lake management.
• Data Scientist: Typically held by individuals with Ph.D. qualifications or similar expertise, focusing on algorithms and predictive analytics.

Having devoted close to two decades to the role of a DBA, I began experiencing weariness around 2018. The toll of being on-call was manifesting as severe health implications. In most settings, DBAs were perceived as reactive problem solvers—earning high remuneration but receiving little attention or respect unless a crisis demanded their intervention. In essence, proactive contributions were unacknowledged, and one’s worth was not recognized as value-added to the business. This prompted me to explore alternative roles. I authored a book for Apress, wherein I engaged with various community members to delve into their roles and responsibilities. After interviewing 29 professionals, I arrived at three viable options:

1. Delve deeper into Business Intelligence (BI) and transition to reporting and BI-related responsibilities. While I had previously occupied a BI role, opportunities in this domain were limited.
2. Cultivate skills in Product Management and transition into a PM or DBA management role. However, the steep learning curve and the prospect of becoming extensively people-focused gave rise to uncertainty. Additionally, I was skeptical of the fervor around Agile methodologies, as I noticed elevated attrition rates among PMs.
3. Embrace a Data Engineer role that primarily involves coding and architecture. Among these choices, this resonated with me the most from a technical standpoint. While I was familiar with R Programming and comfortable with VB.net, I was uncertain whether these skills would suffice. Fortunately, the current job opening aligned with my skill set, enabling me to become a data engineer. Since transitioning, this role has proven to be one of the most fulfilling data-related experiences.

To conclude, the precise definitions of these titles are of secondary importance. The focus should lie on personal aspirations and whether a job aligns with those objectives. Thank you, Josephine, for hosting.