Curious..about data

My dear friend Josephine Bush a.k.a HelloSQLKitty hosts this month’s T-SQL Tuesday. Josephine’s call to us is to share our understanding of the multitude of job titles available out there and what they mean to us.

While many job titles exist, each role’s essence is heavily shaped by the operational dynamics of the business and how responsibilities are allocated among individuals. A rough and high-level comprehension of these titles can be outlined as follows:

• Database Administrator (DBA): Responsible for High Availability/Disaster Recovery (HA/DR), Backups/Restores, security measures, including audits, storage management, code promotions, and sometimes even code reviews. Encompasses Continuous Integration/Continuous Deployment (CI/CD) and automation, and may also be referred to as a ‘Database Reliability Engineer’.
• Database Engineer (DBE): Tasked with managing database code, architecture, and data process flow. Engages in Extract, Transform, Load (ETL) processes, pipelines, and assumes responsibility for data integrity and health.
• Database Architect: Often involves a blend of DBA and DBE roles, with a substantial infusion of cloud architecture expertise.
• Data Warehouse Engineer/BI Architect: Primarily concerned with reporting, data warehousing, and data lake management.
• Data Scientist: Typically held by individuals with Ph.D. qualifications or similar expertise, focusing on algorithms and predictive analytics.

Having devoted close to two decades to the role of a DBA, I began experiencing weariness around 2018. The toll of being on-call was manifesting as severe health implications. In most settings, DBAs were perceived as reactive problem solvers—earning high remuneration but receiving little attention or respect unless a crisis demanded their intervention. In essence, proactive contributions were unacknowledged, and one’s worth was not recognized as value-added to the business. This prompted me to explore alternative roles. I authored a book for Apress, wherein I engaged with various community members to delve into their roles and responsibilities. After interviewing 29 professionals, I arrived at three viable options:

1. Delve deeper into Business Intelligence (BI) and transition to reporting and BI-related responsibilities. While I had previously occupied a BI role, opportunities in this domain were limited.
2. Cultivate skills in Product Management and transition into a PM or DBA management role. However, the steep learning curve and the prospect of becoming extensively people-focused gave rise to uncertainty. Additionally, I was skeptical of the fervor around Agile methodologies, as I noticed elevated attrition rates among PMs.
3. Embrace a Data Engineer role that primarily involves coding and architecture. Among these choices, this resonated with me the most from a technical standpoint. While I was familiar with R Programming and comfortable with VB.net, I was uncertain whether these skills would suffice. Fortunately, the current job opening aligned with my skill set, enabling me to become a data engineer. Since transitioning, this role has proven to be one of the most fulfilling data-related experiences.

To conclude, the precise definitions of these titles are of secondary importance. The focus should lie on personal aspirations and whether a job aligns with those objectives. Thank you, Josephine, for hosting.

This month’s T-SQL Tuesday is hosted by Gethyn Ellis(t).

Gethyn’s Invite to us is to write about the best piece of career advice you’ve received.

One of the most valuable pieces of career advice I’ve ever received emphasizes the importance of protecting oneself and practicing discreteness in communication at work. Some people have this as a natural skill. It wasn’t so for me. I was too quickly triggered and very outspoken during my early years at work. At that time, I held a DBA position at a small company. My intelligent, kind, and diligent boss was unfairly overlooked multiple times for a well-deserved promotion.

I openly supported my boss during occasional town hall meetings where employees were allowed to ask work-related questions to the VP (my boss’s superior). I questioned why he wasn’t being promoted. I can still vividly recall the uncomfortable silence that followed, along with the peculiar looks I received from those around me. The VP didn’t provide an answer to my question, only responding with an uncomfortable stare. After the meeting, most people seemed wary of me and avoided conversation. A thoughtful colleague stopped me in the hallway and offered a short 3-word advice, “Hey, watch your back”.

I was perplexed by their words and wondered what they meant. It also struck me that my boss never bothered to thank me for advocating on their behalf. Soon after that meeting, rumors began circulating that my boss and the VP were frequently seen socializing at various bars and restaurants. Subsequently, my boss received a promotion with significant stock options, while I was disregarded and relegated to less challenging work. I heard from a trusted source that they did not consider me someone who could be relied upon. The guy I spoke up for did nothing to support me. I found another job and left the company soon after.

I have a few similar stories. Learning the value of ‘watch your back’ took me a while. While being outspoken and emotionally honest are admirable qualities that I still I value personally, I’ve learned that expressing oneself without considering the consequences can be imprudent and risky within work environments. The same applies to advocating for others without thoroughly understanding the situation. Employment contracts are akin to marriages, with details primarily between the involved parties and not always suitable for public disclosure. Most people would prefer working things out independently with whatever that scenario may be.

Work cultures vary widely – in smaller companies, there is a high degree of visibility and therefore caution around what you say publicly. The grapevine is tighter, and who are friends with who is observed and talked about a lot. This is partly why people get wary and maintain distance if they see you as the lone person speaking up. They don’t want to be ‘seen’ as being friends with someone openly hostile to upper management.

If you are a team lead or manager, part of your responsibility involves supporting and speaking up for your team members when necessary. However, if you’re not in such a position, it is generally best to focus on your work and avoid unnecessary involvement in others’ affairs.

If you find yourself in an unethical or unfavorable situation(this may include witnessing unfair treatment to others, with all details known), I believe it’s best to consider leaving the job for another opportunity.

This leads me to consider another question. We, from the Data Platform WIT team, are accepting panelist submissions for the WIT panel at the upcoming PASS Data Community Summit. The topic we want to be discussed at the panel is ‘Got your back: Allyship at work’. How does one get to be an ally to others? I believe the first step is evaluating your standing within the company. Are your thoughts and opinions valued and respected? Do they consider you an important and influential person? If you can confidently answer “yes” to these questions, then by all means, offer your assistance. Be sure to take the consent of the person you’re speaking up for and ensure they are okay with you doing it.

Always ensure your situation is secure before taking up someone else’s cause, as it can be unwise to get involved when you’re struggling or merely trying to survive.

In summary, being mindful of protecting yourself in the workplace is essential. Prioritizing self-protection is neither selfish nor cowardly. Tact and diplomacy play vital roles in professional settings, and learning to exercise discretion when necessary can lead to significant rewards in your career.

We all need guidance as we make our way through our careers and our lives. Some people are blessed with the right folks who can offer that by way of friendships and family members. Others have to find it the hard way. This post is about some of my adventures and mis adventures in this regard.

I am a passionate person by nature. I get excited with cool stuff, I go down a lot of rabbit holes, spend a lot of time on something just because I like it. I also come from a culture and a model that believes in ‘sharpening the saw’ and ‘immersion’ on a chosen subject as the only learning worthy of respect and the only model that you do until you retire. My father was a chemical engineer who made metal bearings that go on cars. He started in a small lab, and retired with 3 patents on the topic. Most of us are trained to think similar – to learn any one chosen thing in a lot of depth, invest huge amounts of time and retire experts.

I’ve learned pragmatism the hard way and the importance of attaching utilitarian value to tech I want to master. Those are not my natural virtues; I’ve been burned many times for lack of them.

In mid-2000s, I was a Senior DBA at a large leading healthcare provider company. My core skills were on SQL Server. The MCM had just been announced. It was kind of a done deal among many professionals like me that we’d retire SQL Server experts – and the MCM was exactly what we needed to get there. There were blog posts by Brent Ozar and several others on their adventures getting there. I had just started doing SQL Cruise-s – and during one of those cruises, one of the folks I met and was friends with suggested I take the MCM. Another friend also upvoted that suggestion and said he considered me ‘solid MCM material’, and if I took pains to learn and graduate, it would do wonders for my career. What they said was music to my ears. I went home after the cruise, talked to my boss, and immediately signed up for SQLSkills IE events. I was blessed with an incredible boss and team. This wasn’t hard to do or get approval for. After IE events, I spent a few months digesting the material and reading more. Then, I went to my boss again and requested approval for the MCM exam. It wasn’t cheap. The certs leading up to it were fine, but the exam was close to $2000.

My boss wanted to chat. We went out for lunch, and he asked about my reasons for going for the MCM. I said I was passionate about SQL Server and wanted to learn more. He asked again if I had any plans of leaving or going into consulting. He and I shared a very open relationship, and I would confide in him if I had those plans – I didn’t, so I said I was not looking to move anywhere, just do the exams, and that it would be a good investment in my future. His response was that I was among the best DBAs he had had, that he could trust me to fix any issue or if I didn’t know the solution, I knew the right people to ask. In short, from his point of view, the certification would not add significantly to the value I was providing already. And if it was a personal geeky interest I had – that was fine, but he wasn’t too keen on pushing for approval for the funding. I was disappointed and started considering moving jobs – finding a job that supported the MCM. That problem solved itself – the MCM went away.

During all this, I also figured out a lot of things. I interviewed for jobs that other people were MCMs also interviewing for. In other words, there weren’t any elite jobs you could go to because you were an MCM. Many people who were MCMs felt seriously underutilized at many places they were in. Compared to most full-time jobs, the knowledge was easier to use if you were into consulting (which I didn’t think was my thing). In short, if I had actually gone for it – the geek in me would have been happy, but it would have cost serious money/effort, and I’d have had to face all these realities sooner or later.

• Time is limited. In fact, it is even more limited than money. You can make more or less money. You can do nothing like that with time. Going down rabbit holes or chasing my whims costs time that I am not going to get back.
• Technology is only worth what we use it for. In other words, I could learn a Ken Henderson book backwards, but it will not be helpful unless I apply it to something and turn it around.
• When people offer advice, they mean well, but you are the only person who can figure out what is suitable for you. This has to do with some cultural baggage also for me – in my part of the world, when people who are privileged or at a higher stature than you offer any advice, you *assume* it is right for you and do not examine it further. I still am prone to doing this. A lot of people are like me. It is also deeply ingrained in me, as it is with several people, to seek approval. We sometimes do things that may not be in our best interest to impress the person suggesting it.
  
  Some of the bad advice I have gotten and sent me down rabbit holes, besides the MCM, are below.
• ‘All datawarehousing projects fail. Don’t get into it’. I really wish I had gotten into it – BI would have made it easy for me to make inroads into data science and also visualization/data governance and so on. It is also much easier to learn other data platforms doing BI. And no, that line is ancient. There are many successful warehouses, including at the place I am at now.
• ‘Learning R/Python can get you into data science’ – Nope. I went down this rabbit hole for a while. Blogged furiously on R, read up on statistics, and messed around with R calls from SQL Server. Data Science isn’t as easy as learning two languages. They hire Ph Ds for it for a reason. It is an area that needs solid expertise. You can do some of it with Azure ML and various others, but most data scientists are folks who went to school for data science. And it’s not the coolest data job either. They have their own collection of hassles and mundane work like everyone else does.
• ‘Learning SQL Server is enough.’ – None of the interviews I’ve attended in the past 6-7 years have had serious in-depth questions on SQL Server. They want more cloud hosting experience, containers/kubernetes, open source experience – PostGres/MySQL, on and on. The world has changed since the times of graduating with an MCM and retiring on SQL Server.

So why am I rambling on and on about this? I was reading a blog post written by a dear friend – Tracy Boggiano, on the lack of women speaking on internals at an amazing conference on Internals. This led to a long-drawn Twitter debate where I wondered if mastering Internals would pay dividends in the long run, in today’s world. Many people thought I was wrong in what I said.

My thoughts in this regard were motivated by my past experiences, as detailed above. They had nothing to do with someone else’s passions or imposter syndrome or any other issue. I believe women and anyone should deep dive into what meets their needs as a professional – that could be Internals or anything else, it doesn’t really matter. Take any advice balanced with what suits you, and only you can determine that. If you are prone to the tendencies that I was (getting too passionate, not questioning if advice is right for me and not attaching enough pragmatic value to tech) – be self-aware and extra careful. That is all.

Thanks for reading.

I am writing this blog post as contribution to #NewStarNovember and what got me re-started as a tech speaker in 2020.

I haven’t done a lot of tech speaking – and no the reason is not what you likely think it is..it is not jitters or stagefright or any such thing. I do have my share of nervousness, and everyone should, but that is not what kept me from it. I am a two time Toastmaster, and there was a time when I really used to enjoy speaking.

When I got into the community in 2005-ish, there weren’t too many woman POCs speaking. In fact, there were none. There weren’t many men POCs either, other than a couple of people who were well-known as bloggers/businessmen and also did speaking on the side. At the summit, we saw some MSFT employees who would take to stage to speak on tech topics. Most of the time, they were not experienced speakers. Sometimes, they were not even familiar with the slides they were using – those were made by someone else and handed to them after. The audience rarely liked their accent. Lots of criticism and ridicule happened outside. All of this significantly discouraged me to even try.

Also, I wasn’t someone who dabbled with a lot of tech outside work. I say a lot of’ – because I wasn’t a nine-to-fiver who didn’t touch anything tech after either. I attended plenty of events and workshops, kept myself up-to-date, and read a lot of books. I just didn’t do things like build personal labs and get involved in forum discussions. I had personal obligations and hobbies I liked to pursue in my free time. Why was this important? It was important because without devoting significant time, finding a good topic to present and making myself seen as a speaker was hard. I also set the bar too high here, comparing myself to top-notch speakers like Brent Ozar and Grant Fritchey, and telling myself that I could never be them – if they were speaking, why would anyone care to listen to someone like me?

I had some people ask and encourage me to try – like Kevin Kline and Allen White, but never found it in me to actually try – until Covid hit and in-person events stopped. 2020 was a personally devastating year for me, with several losses. During this time, Ben Weissman and William Durkin started ‘New Stars of Data’. I was working at a great software company and learned some new skills which I thought would interest people. I reached out to Ben, asking if I could submit. Ben could hardly believe that I had not spoken much and readily said ‘yes’, I should submit. I looked for a speaking mentor, and found a great one in Eugene Meidinger. Eugene always found time for me, had great innovative ideas to spruce up the slide deck and watched all my talks. I recommend him greatly to anyone looking to speak, newly or restarting.

That’s how I kickstarted speaking. Ben also paired me with Erin Stellato – someone I looked up to as a speaker and as an ally/great human. That helped me significantly. After this talk, I spoke at 5 events in 2020, 7 in 2021, and 8 in 2022.

I learned a few lessons after I started speaking more. They are as below.

I like speaking but am not obsessed with doing it like some people are. It motivates me, but writing articles and organizing events inspires me even more. I think this is me, and I am not sure this will change.

I do better with allies in the audience. I don’t know if this is a positive or a negative, but I just seem to like it better. When I submit or accept to speak – I make sure my moderator is someone I consider an ally. It just works well that way.

Finding topics is still hard, and the effort to translate an idea into a talk is a lot. I have renewed, massive respect for speakers who do advanced talks. It is hard. Very hard. I plan to work on a couple of talks for 2023 during the holidays.

A sincere thank you to Ben and William for helping me get re-started. I hope my story helps others struggling to find what they like and enjoy doing and learn more about who they are. Thanks for reading!

This month’s T-SQL Tuesday is hosted by my dear friend Kevin Kline (b | t) . Kevin’s call is for us to ‘Tell us the story of how attending an IT conference or event resulted in an amazing career or life opportunity.‘

My professional life literally changed for the better since my first PASS Summit, which was many years ago – I speak of it here. Like Kevin, I was a regular at PASS summit for 21 long years before the organization and the conference in its old form ended in 2020. The fall conference was an integral part of my life – every job I took up during that time came with that condition that the employer support it, and most of them did. My career, in many ways, was significantly shaped by that event and the people I met with there. In 2006, I was awarded the PASSion award for best volunteer. I speak of it here. I did not grow up with a lot of appreciation/recognition. I am from a third-world country where, in my days at least, open appreciation of people, especially women, was rare and unusual. We all have a thirst for the world to know who we are. The PASSion award made that happen for me. On many days when I feel lacking in self-love or respect, I still draw upon that amazing morning when I went up on stage with 4000 people applauding for me. It changed my life.

I have written a lot on SQL Saturdays too – I ran those events for 12 years in Louisville, KY. That helped me connect with a great number of people, and grow many skills with organizing, budget management, negotiation and event planning that I did not have before.

I cannot also fail to mention SQL Cruise a.k.a TechOutbound, which I also attended for 13 years. SQL Cruise was a unique event that gave me opportunities to get to know many people in the community as friends, shed some introvertness and socialize, and also travel and see a number of fun places.

In summary, my entire career has been a series of events that have contributed in many ways to changing my life. They did need commitment and faith on my side though. If you are reading – if you find something like an event that inspires you – don’t stop with attending a random one. Give it your commitment, give it time. Get to know people and help them with an open heart. It can and will change your life. Thanks Kev, for hosting.

I received a great collection of blog posts in response to my T-SQL Tuesday 151 – asking people to write on T-SQL Coding Standards.

Rob Farley (t|b) is of the opinion that coding values that help with efficiency are better than rigidly enforced standards, although he as a consultant abides by his customer’s needs in this regard. ‘Coding standards are not a bad thing. But I think values are better. Valuing readable code, valuing comments that explain the reasons for coding decisions, and valuing patterns that avoid problems, can raise the quality of the coding culture’.
Tom(b) from Czech republic has a neat list of standards they abide by and try to enforce. I particularly liked what they said on specifying nullability on temp tables/table variables, and on code being idempotent – re runnable.

Kevin Chant(t|b) talks about being consistent, and the importance of short, informative comments. Both points are valid and useful.

Aaron Bertrand(t|b) has an amazing post on ‘controversial standard’ CAST Vs Convert – I really enjoyed reading this one. I always use CONVERT personally, but was not aware of the gotchas with CAST that he points out – such as with yyyy-mm-dd and British date, and of CAST being unsupported in other platforms. CONVERT for the win, and am glad I stuck with it. Aaron’s posts are always very informative, and packed with some snarky humor as well. Don’t miss reading this one!

Glenn Berry(t|b) has a nice post where he talks of some basic coding standards, defends his use of NOLOCK and OPTION RECOMPILE on his DMV queries (I believe this is totally fine and use his queries extensively). In the end , he too is of the same opinion as Rob Farley – that ‘it is much more important to have functional, readable code that performs well and is easier to maintain.’.

Deborah Melkin(t|b) has a post that lists the coding standards she thinks are important. She raises an important point – do you fail a PR if a standard is not met if it is not that critical, or just flag it as something to be fixed later? She also discusses the burden of being the ‘one person’ on the team who has to enforce standards and how hard that can sometimes be.

My good friend Steve Jones(t|b) lists a few standards he thinks are important but emphasizes that it is important there is a standard to begin with. He also has a gentle rebuke for me for hosting this blog party 4 times and wants more new folks to start hosting T-SQL Tuesdays going forward. If you are reading this and are interested in hosting, please contact him on twitter.

Deepthi Goguri(t|b) has an excellent blog post on T-SQL practices she wished she knew earlier..it is packed with various tools and tips any T-SQL programmer can use. I did not know of PoorSQL as a formatting tool, and I also appreciated the tip on updating statistics after importing a large number of rows. Well done, Deepthi!

Ken Fisher(t|b) talks of organizational politics with standards – and the importance of having one set of standards everyone in the organization agrees to. I can totally see how this can be a problem in a large organization.

Gerard Jaryczewski(t|b) is writing his first T-SQL Tuesday post – welcome Gerard!! He talks of the importance of simplicity and consistency as T-SQL Standards, naming conventions and adhering to the naming convention of the domain the language uses.

Hugo Kornelis(t|b), an amazing friend I look to for T-SQL related knowledge and guidance talks about having one standard and sticking to it even if you disagree. He also has an interesting take on prefixing view names, something I freely admit I’ve done and enforced as a standard many times..Hugo believes tables and views have to be interchangeable, so you don’t need to be named differently and there are other ways to find out if something is a view..my take personally has been that view is considered code while a table isn’t..so knowing what is code helps in including it for various purposes of analysis. Regardless I respect Hugo’s stance and will consider it going forward for my standards too.

My good friend Tracy Boggiano(t|b) has some great tips on readability here. She also suggests putting performance ahead of any formatting related standards.

To summarize, the lessons I learned are as below –
1 It is important to have standards
2 Pick one set of standards and stick to them.
3 Respect standards at the client (if you are a consultant) or existing standards wherever you go.
4 Performance and code that functions right is more important compared to how it looks, although readability matters too.
5 Standards include stuff that you shouldn’t use, like CAST instead of CONVERT.
6 Many other standards are buried in the posts above.
7 Sign up with Steve to host a T-SQL Tuesday, if you haven’t already!!

That is it!! Thank you to everyone who participated and sharing your nuggets of wisdom.

T-SQL Tuesday is the blog party started by Adam Machanic (b|t) over a decade ago and is now maintained by Steve Jones (b|t) on tsqltuesday.com. On the first Tuesday of each month, a blogger hosts the party and suggests a topic. Anyone interested in the topic gets to blog about it. You have one week to write your blog post. It has to go live on the second Tuesday of the month. The host then compiles a list of posts and creates a blog post, for a summarized read. It is a great way to get thinking on subjects you may not have thought of, read other people’s ideas on the same subject, and also get a few people to visit your blog to read your thoughts. Many bloggers including me have used this to get started.

I am honored to be the host for June 2022’s T-SQL Tuesday.

My invite is about coding standards, or what I now call Linting Rules, for T-SQL. What are the T-SQL coding rules that are appropriate for where you work and what you do? If there are exceptions to those rules, state them too! If this is enough, read the blog party rules below and get started!!

• Your post must be published on Tuesday June 14, 2022.
• Your post must contain the T-SQL Tuesday logo (see above) and the image must link back to this blog post.
• Trackbacks should work, but if not please put a link to your post in the comments section so everyone can see your contribution! (My comments are moderated so please don’t worry if yours doesn’t appear right away, I will make sure it does!)
• If you are on twitter include the hash tag #tsql2sday – it helps with RT-s and visibility!!

More on why I picked this topic as below –

When I started out as a DBA two decades ago, I had a list of rules that I would carry with me into every job I went..these are things I look for in T-SQL code and try to enforce as standard. Some examples were casing rules, minimized usage of SELECT STAR, equating the right data types in columns, avoiding NOLOCK hint and so on. Standards ensure quality and consistency in code.

Standards differ for each firm, depending on what is appropriate for an environment..it is even possible to have varying standards in the same company, depending on the environment and what is appropriate for a database. This is an excellent article on what are the different components that comprise coding standards, and why we need them. I am also a big proponent of automated code checking for standards – there are lots of tools available for doing this – SQL Prompt, which is a personal favorite of mine, and many others as listed here.

Several tools currently do linting on many relational platforms, not just SQL Server. Almost all of them though, have rules that the author(s) think are best for the worlds they work in, and do not include other conditions which they have not encountered yet. A common example I like to use is unnamed primary keys on temporary tables. There is nothing inherently wrong with having an inline primary key constraint/index on a temporary table – but if you use Query Store, plan forcing on a plan that uses this temp table will not work simply because the constraint gets named differently each time. When I started to look for a linting tool for where I work – I ran into so many rules that were non-existent or not applicable to my environment with outside tools that I decided to write my own using ScriptDOM – a Microsoft-provided library that was created specifically for this purpose.

It would help greatly if we had a collection of rules that people use to pick from and enforce as appropriate for their environments. It will also help me to code some of these into ScriptDOM and put it out on GitHub, if the rule is one that ScriptDOM can find easily. So, re-stating the call for this month – What are the T-SQL coding rules that are appropriate for where you work and what you do? If there are exceptions to those rules, state them too!