T-SQL Tuesday #97 Roundup: Learning goals for 2018

SqlTuesday

T-SQL Tuesday is a monthly blog party started by respected SQL Guru Adam Machanic. I was honored with hosting this month’s party. I picked a topic that is important for me at this time of the year – setting learning goals. I was surprised and somewhat overwhelmed to see the number of responses. I learned a lot reading what every person had to say. Below is the summary.

1 Rob Farley talks of tuning learning to what customers look for – which is a good idea for most of us. He also explains being interested in certifications and Microsoft EdX course for data science. This seems to be a certification many people are interested in. (Rob may not need as much ‘learning’ to do it as some of us as he is one of those who passed MCM without a lot of prep 🙂

2 Andy Leonard talks of learning more on cloud based offerings. He also explains challenges with managing businesses, home-schooling children and various challenges it presents with managing time. Andy is considering getting up at 5 AM to find time to learn – I must confess that I went ‘wow’ reading that because am not a morning person and that speaks for so much dedication. There is no shortcut to hard work and here is a prime example.

3 John Deardurff talks of finishing what he started – John is the certification guru of #sqlfamily, he has every cert imaginable under his belt and has listed a few more to go, in addition to finishing a college degree. Again, nothing short of inspiring.

4 Bert Wagner talks of learning more DBA skills as a developer to architect solutions better. Bert has also included a fun little video talking of his goals, very worth watching and very creative. He mentions the importance of creating demo-s as a way of learning.

5 Blogging rockstar and my own personal inspiration to blog – Brent Ozar talks of learning and creating  gamification techniques  around technology design such as cloud deployment strategies. Gamification techniques are intended to leverage people’s natural desires for socializing, learning, mastery, competition, achievement, status, self-expression, altruism, or closure, or simply their response to the framing of a situation as game or play. (Yes I had to google that and read up on what it was!). I can’t wait to read more of what he comes up with in this regard and I’ve always found his ideas to be incredibly creative.

6 Michelle Haarhues talks of learning more on Microsoft Azure and passing related certification exams. She compares tech learning goals to fitness goals and likes learning at Starbucks compared to home. She ends her post with a well-worded crisp mission statement on her goal.

7 Jeff Mlakar talks of one of my favorite topics – time management. His post includes a very good video-lecture on this topic. It is very worth watching. Jeff’s goals include MCSE, blogging,speaking at various sql saturdays and networking.

8 Steve Jones, one of my best friends and mentors in the community talks of various challenges that come up with setting structured goals in general, following a book or a course to the end (his experience with powershell is almost exactly the same as mine), and tracking goals. Steve’s post resonated with me in many ways.

9 Deborah Melkin aka Deb the DBA talks of learning better automation of testing techniques and her own style of learning – visual and kinesthetic. She explains the importance of matching how we learn to our learning style and has also included a link on learning styles.

10 Chris Voss talks of completing his MBA program and focusing on skills he needs as a data architect. He mentions the importance of DBA fundamentals as well as using the SQL community slack channel.

11 Long time sql community member and security guru Brian Kelley talks of maintaining MCSE and CISA certifications, as well as doing the data science certification.  He talks of the difference between just getting a certification and gaining knowledge as part of the process.

12 Garland McNeill talks of list of skills to gain as a DBA and the importance of working complex problems as a way of assimilating the learning.

13 Tracy Boggiano talks of being inspired by Andy Warren’s talk on ‘Building a Professional Development Plan’ at the summit. I’ve heard Andy deliver this talk too and have been similarly inspired. Her goals including getting better at unix, talking at more events (she has spoken at 18 sql saturdays last year – well done Tracy!) , and learning more on SQL Server 2017.

14 Marek Masko talks of earning MCSE on cloud platform, speaking at double the number of conferences than he did this year and attending local events in Poland. He also has a great list of books on database engineering that are worth considering for any of us.

15 Taiob Ali’s goals include Microsoft Big Data Certification,Architecting Azure solutions and learning more on powershell. He also wants to contribute more to #sqlhelp answers and stack exchange.

17 Melissa Connors from SentryOne writes on learning more on documentation and communication – she includes links to courses on EDx related to this, it is a very interesting list. She also includes some pictures of getting to her goal of baking macaronis (yum!) and training her dog Trekker.

18 Samir Behara’s goals include technical certifications and learning more on cloud technologies. He has made an interesting analysis of correlating website traffic and speaking engagements, showing that speaking increases visibility to the blog as well.

19 Glenda Gable’s goals include getting technical certifications, learning more using Brent Ozar’s training, blogging and using MSDN for labs. Glenda is also interested in being partners for accountability towards our goals.

20 Todd Kleinhans wants to learn more on Linux, Hekaton and do the Microsoft Data Science program, in addition to a number of personal goals. Todd has a lovely habit of writing hand written notes of thanks or appreciation for any number of things and posting it on social media. He did one of this blog party too which I will share towards the end.

21 Former PASS Community Evangelist and BI Architect Cathrine Wilhelmson talks of her goals that include learning more BIML, speaking, blogging, teaching and researching. She wants to revisit her post in December too to find out how she did.

22 Lucas Kartwidjaja talks of learning more of AWS/Azure via online courses and also do more of cloud architecture.

23 David Hiltenbrand has many interesting things to learn on his list including CosmosDB, graph databases, Adaptive Query processing via several online courses. He also plans to attend SQL Saturday Nashville (hope to see you there David!).

24 Martin Catherall from down under talks of general trends around learning and the importance of learning something deeply.

25 Stuart Moore blogs about doing more networking, improving his writing skills, doing more automation (everybody should!) and getting better at marketing himself. I could see eye-to-eye with every one of those goals myself.

26 Doug Purnell has a very interesting link on tendencies when it comes to goal setting – there is a test there that seems informative (mine came up as ‘questioner’, which I am) – I guess one needs to read the book he recommends to find out how this helps in setting a plan and making it work.

27 Andrew Tobin writes on getting into a new job that is 100% sql server, learning more related sql server stuff online, blogging without ‘getting personal’, answering questions on Stack Exchange and doing videos on performance tuning. Good luck Andrew!

28 Shane O Neill uses a joke to explain the many things he has on his list to cover next year including Azure, CosmosDB, containers, python and so on. It is a really creative way of presenting a plan and fun to read.

30 Jason Squires talks of many things he wants to learn including XEvents, Powershell and  Availability Groups – he also includes his goals to improve on blogging and speaking at online events.

32 My good friend and mega blogger Kenneth Fisher writes on his own ‘eclectic’ learning habits and includes cloud,t-sql,performance tuning and learning more on metadata. Ken recommends starting to blog slowly to build up momentum – a great tip for new bloggers.

Thank you to all of you for taking time to contribute. I like the suggestion made by Glenda Gable on being partners to help with accountability on our goals. If anyone feels up to this just leave a comment below, we can set up a slack channel or something similar to help each other.

Merry Christmas and Happy New Year!

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

T-SQL Tuesday #97 – My learning plan

SqlTuesday

This month’s T-SQL Tuesday is hosted by…er..yours truly. I picked a topic that usually comes up for me every December – learning goals for the next year. This is the first time that I am blogging about it.

Usually this review is accompanied by a strange combination of imposter syndrome, guilt and thoughts like ‘how about not having goals’,’who cares’,’I want to retire’,’I am never going to be as good as so-and-so’ and so on. In short, it has rarely been a pleasant experience.  A lot of people me included, want to learn a lot in the little time we have. We underestimate other life challenges that get in the way, and also our own power to assimilate and actually use what we learn.  We also have different degrees of focus and methods of learning that work for us and there is no one fit for all. This year I decided to be more pragmatic and take all of those things into consideration while making a learning plan. I took into consideration below:

1 I can devote about an hour a day to learning on a normal day – this excludes days when I am on call and have had a late night, or some kind of a personal stressful situation.
2 I learn best by in person classes, followed by podcasts and videos.
3 My areas of interest are wide – DBA work with BI/Data visualisation. I need to pick what I want to learn carefully.
4 I consolidate well when I apply at work(true for almost everyone), teach/present and blog.
5 I network better in small crowds and familiar places.
6 I usually walk for about an hour in the evening. I can use that time to listen to podcasts.
7 My ‘lab time’ is the time I spend actually putting what I learn to actual practice – is typically about 2-3 hours late Saturday afternoon.

Keeping all this in consideration, my plan is as below:What to learn:My areas of focus are as below:

  • SQL Server: Azure, Powershell and Extended Events
  • Non SQL Server: CosmosDB or MongoDB, Beginner Level Python, Mid level R
  • Data Visualisation: PowerBI
  • Certification: Microsoft Professional Program in Big Data


How to learn:

1 Pluralsight will be my main tool to begin with since it is easy and affordable. The goal is to try and get in 1 hour of pluralsight training per day on good days.
2 I will be attending select sql saturday precons as a second option, particularly for Azure and Data Science related training. I’ve set aside a budget for this and will only be able to do those events that fall within the budget. Chances are that none may.
3 I will be watching PASS Summit recordings and Ignite recordings for additional info. I don’t consider either of these as in depth learning but more as informative sessions.
4 I will be using Edx for the Microsoft Professional Program in Big Data.
5 I will be attending TechOutbound (formerly SQL Cruise) in March and the PASS Summit later in the year for networking. Additional networking will be possible with sql saturdays if they happen – most definitely with my own sql saturday event in July.
6 I found a treasure of podcasts that are out there that I can listen to on my evening walk. My favorite site for them is sqldownunder, by Greg Low. But there are several others – list here. These are my insurance that I get *some* learning in even if all the rest of the stuff does not work out. Listening during exercising is easy and exercising is something I do with a passion.

How to apply:
1 I am somewhat limited in terms of how much I can travel, for many reasons. So, I will be presenting at several virtual chapters and also user groups that allow remote presentation. Right now the goal is 5 presentations, more depending on time.
2 I plan to blog at the rate of one blog post per week. I’ve come to this as a result of a lot of experimentation and this works well for me. Every other week will be a non technical post since I have a lot of non technical info to blog about.
3 I do not consider application of learning to work as something I can blog about, so not writing more on that.

I also plan to journal weekly on how I did in the past week and tune myself as I go.By ‘journal’ I don’t mean writing pages – just a simple excel sheet where I review what I learned versus what I wanted to learn. If I go for three weeks without meeting any goals I am doing something wrong and need to re adjust. Am hoping to keep my pace with this better and do better as I go.  I think this is vitally important. It is not a bad thing that we failed to meet our goals but it may be bad if we never even considered tracking our progress through an entire year.

One of my favorite movie scenes is from ‘Gone with the wind’, where there is a garden clock with the lines ‘Do not squander time, it is the stuff life is made of’..right next to it is a cat sleeping blissfully. That sums up time management in general – you can’t force it. You have to take little steps, be consistent, and factor in all the other things that go into how you use your time. I hope to be better at it this year than I was in the years before.

 

 

 

 

 

 

 

 

 

T-SQL Tuesday #97 – Setting learning goals for 2018

 SqlTuesday
This is my first opportunity hosting a T-SQL Tuesday and am super excited!!
T-SQL Tuesday is the brainchild of well respected SQL Guru and author of ‘sp_whois active’ – Adam Machanic (b|t).  Adam rightly predicted that we all could benefit from ‘a recurring, revolving blog party’ with a new topic given each month – the party has been on since 2009 with great benefit to bloggers old and new.

This month’s T-SQL Tuesday will take place on Tuesday, December 12, 2017.

It is the end of the year…each person has their own way of calling it a year. For many it is time to take those leftover PTO days, enjoy the time with family and friends, and be grateful for the many gifts we are fortunate to have received. It is also a time when we have to ponder the year ahead and how we plan to use this time in the best manner possible. For many years I never consciously considered setting or pursuing learning goals. I just learned what I needed for my job. If there was a new version of SQL Server out – I went after learning new features in it. But today – it is no longer possible to be that simple. Also, it is very difficult to focus on anything unless you put it down as a goal. One of my favorite quotes by Zig Ziglar sums it up best ‘ A goal properly set is halfway reached’.
Learning more on just SQL Server is no longer enough. We need to learn other tools and technologies. There are many of them. There are 3 things to address to me when it comes to goal setting with technology and learning –
1 What do you want to learn? (specific skills and talents)
2 How and when do you want to learn? (methods of learning and timeline on learning)
3 How do you plan to improve on what you learned? (Putting it to use at work/blogging/speaking)
I will explain each of those in detail below.

1 What do you want to learn?

This varies depending on your line of work and where you want to go career wise. I will give a few examples.

  • If you are into learning about the cloud and hosting – you need to know what options are (on AWS and Azure, to begin with). Also on multiple other smaller/private hosting providers. You need to know how to transfer data/how much it costs to scale/can you turn it on and off as necessary…any number of things.
  • If you want to learn other non SQL database platforms you’d have to think about which ones are important to you – postgres, CosmosDB, DocumentDB or even MYSQL or Oracle.
  • If you plan to get into data mining and analytics – there are several things to learn in that area. I just started getting to intermediate level with R , and now we have Python that works just as well with SQL Server. You are also better off learning other skills that go with data mining – such as cleaning data, setting up the solution on an ongoing basis and so on.

 In general it would be wise to narrow your focus down to your areas of interest and pick a few things – not too many but perhaps 2-3 things you’d like to focus on and get some depth of knowledge in.

2 How and when do you want to learn?

After you get those goals in, how do you plan to get the said training?

There are countless options, with time and costs to consider. The cheapest ones are Ignite videos (for free), Pluralsight subscription (30$ a month), EDx/Udemy courses (all reasonably priced).SQL Saturday precons (very reasonably priced day long training) as well as SQL Saturdays themselves(free day long training on saturdays). If you can afford it yourself or work at a company that pays for training – consider Tech Outbound (formerly SQLCruise) or PASS Summit.

There are  networking goals to consider as well. I personally would never have thought of networking as a ‘goal’, am able to tweet or message most folks and talk to them, so what is the big deal? No. Meeting people in person is a whole different thing, and you never know what doors that can open. Networking goals can be like meeting 10 people new (some people set them that way), or catching up with 50 people you already know including 3 lunches with people who have most regard for. You’d have to consider where and how you are going to get those goals met. For some people, like me, this is not a numbers game – I’d just like to say am going to be at Event A, B and C and do my networking there. That is totally fine too.

3 How do you plan to improve on what you learned?
  • The primary application of knowledge is at work. You want to think of upcoming projects or opportunities to apply this knowledge. For most people this comes up at a performance review that happens early in the year. Many people are also not comfortable making it public. If it is not bloggable that is ok – but if it is bloggable do consider sharing it. 
  • What are the chapter meetings, events you plan to speak at? If that is too much detail, consider how many of those you’d want to do.
  • What is the frequency of blogging you’d like to maintain?
  • Are you planning on writing books or coauthoring any?
  • Are you  planning on participating in forums to answer questions – such as on SQL ServerCentral.com?
  • Are you planning on any other group contribution – such as Idera’s #sqlchat on twitter or even answering #sqlhelp questions on twitter?
  • I’d put certifications and tests too in this category as they give a name to what you learn and add a credential.

So, that is quite a lot to think and write about. Below are the rules of the game when you are ready:

  • Write your post.
  • Schedule it to be published at some point on Tuesday, December 12th (UTC calendar day).
  • Include the TSQL Tuesday logo in the top of your post.
  • Link your post back to this one. Trackbacks should work, but it is a better idea to write a comment with the link on this post. Please give me some time to moderate your comments.
  • If you are on twitter, tweet a link to your post, adding the #tsql2sday hash tag. Most people filter by hashtag and it increases the visibility of your post by quite a lot.

Look forward to reading your goals and learning from them!! I will publish the summary on December 19th 2017.

TSQL Tuesday #96 – Folks who have made a difference

SqlTuesday

This month’s T-SQL Tuesday is hosted by Ewald Cress, and the topic is ‘Folks who have made a difference’. This is an opportunity – and right around thanksgiving – to give a shout-out to people who have made a difference in our careers and in the data community. To me this list is so long that I would need several blog posts to write about every person on it. But am going to pick one person, and for a very good reason. That person is Andy Warren(b|t) – the founder of SQL Saturdays.
Back in 2008, I was actively running the local user group here at Louisville, and volunteering for PASS in several different capacities. I was always looking for other opportunities to bring training to the local community, that was more than one hour talks we had at user groups. Part of the reason I wanted to do this was because the vast majority of people I worked with and attended user groups were folks who had never attended any kind of formal training.Very few companies paid for them to go, and the idea of investing your own time to learn was very new, back then. You learned what your boss wanted you to learn, and if he gave you time/money for it. That was the general attitude, and one that struck me as very wrong, although I didn’t get very far trying to say that to most people. I met Andy at the networking dinner that he and Steve organized, at the summit. He asked me if I had heard of SQL Saturdays. They were free day long trainings, and could be held anywhere. It was interesting, but I was apprehensive. People attending training on saturdays? In a little town like this? I wondered about it and when I got home – I called my contact at a local school on a whim, just to find out if they had any classrooms they could lend to us for free on a saturday. The answer was a very emphatic ‘yes’, of course they could. Their rooms were small but they had 4 rooms, plus a nice big hallway, and small private rooms too. They’d like the exposure in community and actually wanted to do it. This looked like something that wanted to happen.
I registered on Andy’s site, picked a date and then looked to him for guidance on the rest. He guided me through it every step of the way, with constant reassurance that ‘if you build it ,they will come’. SQL SAturday #1 was event #23 for PASS. It was held at New Horizons, with 60 people in attendance(I expected about 40 people), 4 speakers(Arie Jones, Allen White, Kevin Kline and Dave Fackler) and two sponsors. I think we had about 3k as sponsorship money to run it. Almost every attendee said emphatically about how much they loved it. New Horizons wanted to host it yet again. The upcoming year, 2018, will be our year #10 running SQL Saturdays, in a row. Every year, we have been getting bigger and better. Hundreds of people have received free training, so many careers and lives have been touched.
I would never have done it if it was not for Andy talking me into it. I think every one of you who has been part of SQL Saturday Louisville need to be. Thank you Andy, for showing us what we could be.

Dealing with disquiet

Last week I listened to a podcast from one of my best friends in the sql community – Kendra Little. In this podcast Kendra talks of her encounter with anxiety attacks and how she dealt with them. When I listened to her honest, moving story – my mind was filled with thoughts on the many episodes with anxiety i’ve dealt with, each of its own kind. I was also moved by how many people in the community wanted to hear about stories like this to feel reassured on their own journey. So, here is one of my own. I am tagging a friend at the end of it ,hope he will tag someone else, and we can have a worthy collection of stories to refer to if we need help/reassurance.
Many years ago, I worked as a Senior DBA cum team-lead at a big firm. I was doing DBA work, and also helping my boss manage a team of six other people. My boss was a very kind,intelligent,generous man, one of the best I’d worked for.I greatly enjoyed my role and the work.A few years down the line, my boss got passed up for a promotion he richly deserved. After that his attitude and behavior changed.He started to be a no-show at important meetings,didn’t respond to emails, took time off without notice, and so on. One day, his boss decided to forward one of his meeting invites to me.I went, and filled in for him.The next day, I got more of his work. And then more. Soon, I was doing two people’s work, and working 12-14 hours a day. I wanted to speak to someone about this, but I kept putting it off with the hope that my boss would come around and get back to doing his stuff. I still really liked the job, and kept up with my needs for food, exercise etc too with the demanding schedule. At least I thought I did. One day, I started to feel some pain around my shoulders. I rubbed some balm on it and hoped it would go away. The next day, there was some tingling sensation in my feet, followed by some numbness and brief giddiness. I started to become very jittery and noise-sensitive. Somebody honking on the street would bother me for hours, with my hair standing on end and my heart beating extra loud. I had never had these symptoms before, and as Kendra mentioned – my life was going well according to me. So, what was wrong?

After a few days into this – as I was driving to work one morning, there was more loud honking on the busy street I had to use. My whole body was thoroughly shaken. Instead of going in to work, I drove myself to the ER – firmly convinced that I had some strange disease. They did all kinds of tests on me – brain mri, abdominal CT, heart exams, everything – pronounced me fine and sent me home in two days, with some medication to help me sleep better.

Two days later, I went in to the dentist for an unrelated problem with my wisdom tooth, still worried and firmly convinced that I had some unknown illness. While taking xrays of my teeth, the dentist said he noticed that my jaw bones were not aligned – a condition called TMJ. I asked him about my symptoms, and he nodded yes, I had TMJ , one of the major causes is stress and can be treated. This diagnosis was followed by a visit to a jaw specialist, some braces to wear and LOTS of relaxation therapy/counselling. After 2 months my ordeal was finally over. My TMJ still comes back  now and then to remind me that I overwork or am not taking enough care of myself. But I know how to handle it now. Needless to say I moved on from that job shortly after.

Below are the lessons I learned from that episode and what I follow as practices for mental (and physical health), dealing with stress and anxiety.

1 Respect your body – Your body is an entity of its own. One of my friends likes to joke that you are its boss before 50 and it is your boss after. That is mostly true (you are never its ‘boss’, it just cooperates better when it is younger). Your body does not care how much you love your work or how long you want to do it. It is undoubtedly true that liking what we do leads to more mental happiness, it is not true that it is a safeguard against self care. Avoid the line ‘I love what I do’ as an excuse to skip meals/skip exercise/not getting enough sleep/not taking vacations or less family time. It is not worth it and can hurt you, a lot.

2 Connect with spirit – Kendra talks of this as going back to church/community where she could find succor/replenishment for spirit. I don’t particularly care for community worship, for many reasons. To me, connection with spirit happens with doing things that bring me joy – reading books I love, particularly books on women’s empowerment(‘Women who run with wolves‘ is my favorite), visiting book stores, antique malls, doing gardening, drawing or painting. Spirit is available in a place that is safe and free of rules, and those are the spaces for me.

3 Practice personal reassurance –  I believe each person needs this in their own way. To me seeing art I enjoy , favorite pictures from vacations/with friends/family, or some phrases that resonate with me are very reassuring when I get anxious. I keep good art all around me, and phrases from books like ‘Tao of Pooh’ or ‘Jonathan Livingston Seagull’. I also invest money in enlarged prints of photographs taken on family vacations or sqlfamily reunions around me. They serve to enrich how I feel through the day. I change them around periodically but make sure that I see them – not just give passing glances but really ‘see’, standing in front of them, and re-live pleasant, happy moments.

4 Practice deep breathing and guided meditations. I recall a quote I read long ago – ‘Slow breathing is like an anchor in the midst of an emotional storm: the anchor won’t make the storm goes away, but it will hold you steady until it passes’. I practice it whenever and wherever i can, with my hand on my belly, where I feel my breath the best. It calms me down like nothing else. As a sound sensitive person, I love meditations that come with bilateral stimulation – a scientifically proven way to relax your brain. One of my favorites is here. Another awesome one called soft-belly meditation is here.

5 Understand your triggers and work with them – majority people who have anxiety have to deal with it periodically, it never really goes away fully. It teaches lessons in self acceptance that are invaluable. To me – I am triggered by loud noise, heavy traffic,  noisy crowds and certain argumentative/demanding situations. And, as I learned from this particular anxiety episode, I need to find time for self reassurance. I don’t accept or work jobs that do not leave me time to balance these aspects of my life. As writer Stephen Covey says, that becomes like driving without finding time for gas. The car is going to stop, whether you like it or not.

Last but not the least, get help when your body reacts in ways you do not understand. I cannot stress this enough. Sometimes help is the first doctor you go to. Sometimes it takes longer and needs different types of doctors/healers/therapists/techniques. But persist. We live in an era where so much information is available online – search for information, ask friends on facebook or other social media on what they think.  I am personally very grateful for the many people I have befriended because of the issues I have had – kind, sensitive, beautiful people who have taught me the value of life and importance of living in the moment. I hope to be the same to anyone who needs help with anxiety, stress or similar.

I tag one friend here – Tim Costello – to narrate his story. Tim, pass it on to someone else after you, and thank you.

“The most beautiful people we have known are those who have known defeat, known suffering, known struggle, known loss, and have found their way out of the depths. These persons have an appreciation, a sensitivity, and an understanding of life that fills them with compassion, gentleness, and a deep loving concern. Beautiful people do not just happen.”

– Elisabeth Kubler Ross

 

PASS Summit 2017 – the best ever

I have been a PASS summit attendee for 13 years now. This year is my 14th. Every year is different – some years are better than others, some make you feel it wasn’t as good as usual. This year was an odd experience for me. Quite a number of my friends were not attending for various reasons, and there was a lot of content that I thought was not in my line of work. In short, I went because I always do but didn’t expect a whole lot. But, it didn’t quite turn out that way. Some of my highlights are as below:

Dressing up for Halloween: Had a good day at precon on Data Science essentials followed by a quiet dinner with a good friend and headed home to the airbnb which I shared with two other friends.I had bought a halloween costume along.Now, am not normally much of a costume person – I bought this costume (‘Maleficient’ the evil queen from Sleeping Beauty) at the very last minute – because I had some amazon points left and amazon in its own clever way thought I’d like this costume because I like Disney movies. I thought this summit was going to be very low key and decided that dressing up would probably make it more fun.So in went the costume in my suitcase, literally with prime wrapping intact.In the lodging – I saw my good friend Mickey dress up for halloween. She looked fantastic and the girl in me wanted to dress up as well. So I put on the costume, which actually fitted me right, and came with a lovely glowing staff. And off I went, first to the WIT dinner and then to the opening event. Everyone I met loved the costume and by the time evening ended I was beginning to tire of how many photos I had to pose for. The part of me that looks for attention had gotten more than her fair share during this one evening.

Making new friends: I am normally a somewhat reserved person. It takes me time to warm up and make new friends – part of the reason why I love the summit is because I know so many people there and they know me just by virtue of attending. There is no extra effort to stretch out and make friends. This year, I just decided to push the part of me that sinks into this comfort zone. I went out and made friends with people who I felt were worthy being friends with, and especially those who were looking to be part of the community. Those worthy contacts include Miyo Yuk, a data scientist from MIT, Swagatika Sarangi, immigrant from my home country and new speaker, and several others. I also met with new comers I was mentoring and had a very good conversation.

Asking for help/being mentored: Also a very difficult thing for me – when I do it I do it in very awkward ways that do not get desired result. This time I think I got it right. I got some awesome mentoring advice from two gracious ladies I have great regard for – Kathi Kellenberger on writing books, and Jen Stirrup on WIT related issues. I am glad to have reached out to both of them.

Learning data science: I have been blogging a while on some basics of data science. At the summit I attended some excellent sessions on how the data world is changing and evolving, what are the areas I can specifically focus on as a SQL Server professional looking to do more of data science related work, and who are the people I need to follow for getting there. I felt more energized than I ever thought I would that going in this direction would be the right thing for me, although it does involve a steep learning curve.

For all the reasons cited above and for many others, this summit was special. It has always been special but this year I felt a sense of true belonging with people in a very obvious way. The feeling was strongly like ‘this was it’. These are the people I am going to be with and grow old with through the rest of my career and my life. I am grateful and glad to know so many of them so very well.  I am grateful that we are linked by a common worthy cause. If you are like me and reading this  – looking for a community to belong and friends who support you/care for you – take heart, you have arrived. Just give it time and give it your whole hearted commitment. It will pay off.

True belonging is not something you negotiate externally, it’s what you carry in your heart. It’s finding the sacredness in being a part of something. – Brene Brown

 

 

Understanding ANOVA

ANOVA – or analysis of variance, is a term given to a set of statistical models that are used to analyze differences among groups and if the differences are statistically significant to arrive at any conclusion. The models were developed by statistician and evolutionary biologist Ronald Fischer. To give a very simplistic definition – ANOVA is an extension of the two way T-Test to multiple cases.
Using the same dataset from previous post – the chicken feed one, let us analyze this further. The inference from the boxplot drawn in previous post was that the weight of the chickens is lowest when they are fed horsebean and highest when they are fed casein or sunflower. There were also lots of overlaps in weights. Our objective here is to determine if the average weight of chickens is significantly different based on feed, or is it more or less the same, which means the feed is really not that important to consider? We will define our null hypothesis(the statement to prove or disprove) as the average being the same and hence type of feed has no real consequence.
Null Hypothesis:H0: There is no correlation between feed type and weights.
Alternate Hypothesis:H1: There is significant correlation between feed type and weights.
What ANOVA does is calculate F statistic = Variation among sample means/Variation within groups. The higher the value of this statistic, the greater is the chance that variation among sample means is significant.
Running this simple test on chickwts dataset as below:

anova

From this we can see that the F value is 15.365(Large is > 1), and the p value is really really small(to remember that ‘small’ is <0.05). So we can say with confidence that difference in weights between different feeds is way higher than difference in weight within same feed. In other words feed does appear to have an impact on weight. So we accept the alternate hypothesis.
Taking this one step further – what are the types of feed that cause significant weight differences? To understand this we perform what is called a Tukey’s HSD test, that compares each value to every other and helps us understand which pairs are significant.

It just takes a couple of lines of R code to do this – as below:

anova1

How to read/interpret the results of this test? Let us take the first line. The difference in weight with horsebean and casein is -163, which means casein is 163 points above horsebean. Since the p value is 0, the chances of this being significant are really small, as we can see with lower and upper limit values. So this is really not the pair we are looking for. Going down the list, the ones with significant p values (> 0.05) are meatmeal-casein, sunflower-casein, linseed-horsebean, meatmeal-linseed, soybean-linseed, soybean-meatmeal and sunflower-meatmeal. This can also be drawn in graphical form as below (i could not get R to shorten the text names but the ones beyond 0 are significant). The pairs with significant differences are the ones worthy of pursuit on which feed to adopt. Thanks for reading!

anova3

 

 

 

 

 

Box-and-whisker plot and data patterns with R and T-SQL

R is particularly good with drawing graphs with data. Some graphs are familiar to most DBAs as it has been things we have seen and used over time – bar charts, pie diagram and so on. Some are not. Understanding exploratory graphics is vitally important to the R programmer/data science newbie. This week I wanted to share what I learned about the box-and-whisker plot, a commonly used graph in R – and one that greatly helps to understand and interpret spread of data. Before getting into specifics of how data is described with this plot, we need to understand a term called Interquartile Range. For each range or subset of data we are involved with (or rather the field we choose to ‘group by’), the middle value is called ‘Median’. The middle of the top 50% of values is called first quartile, and middle of the bottom 50% is called ‘second quartile’. Difference between first and second quartile is called ‘interquartile range’. A box-and-whisker plot helps us to see these values visually – and in addition to this also shows outliers in the data. To borrow a graphic from here – it is as below.
inqr

Now, let us look at the dataset on chicken weights in R with the help of this type of graphic.This is basically a dataset comprising of data on 71 randomly picked chickens, who were fed six different types of feed. Their weight was then observed and compiled. Let us say we sort this data from minimum to maximum weight for each feed.

You need the mosaic R package installed, which in turn has a few dependencies – dplyr, mosaicData,ggdendro and  ggformula. You can install each of these packages with the command install.packages(“<package name”, repos = http://mran.revolutionanalytics.com&#8221;) and then issue below command for pulling up with B-W plot.

library("mosaic")
bwplot(weight~feed, data=chickwts, xlab="Feed type", ylab="Chickem weight at six weeks (g)")

Results are as below:

chickfeed

With this we are easily able to see that
1 Feed type casein produces chickens with maximum weight
2 Feed type horsebean produces chickens with minimum weight
3 Feedtype sunflower has some outliers which don’t seem to match general pattern of data
4 The distribution of weights within each feed type seem fairly symetric.
5 There are also many overlapping weights across feed types.

To get to some of the numbers accurately you can just say

favstats(weight~feed, data=chickwts)

rstats

If you wish to get some of these values in T-SQL (i imported this data into SQL server via excel) – you can use query below. Comparing the values visually with graph will show that they are similar.

SELECT DISTINCT ck.feed,MIN(weight) OVER (PARTITION BY ck.feed) AS Minimum, Max(weight) OVER (PARTITION BY ck.feed) AS Maximum, 
AVG(weight) OVER (PARTITION BY ck.feed) AS Mean,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY ck.weight)OVER (PARTITION BY ck.feed) AS MedianValue,
STDEV(weight) OVER (PARTITION BY ck.feed) AS SD
FROM chickfeed ck ORDER BY feed;

Results are as below:

sql

The math for Q1 and Q3 are a little complicated in T-SQL, I was unsure if it was worth doing since that is not the point of my blog post. You can find info on it here

As we can see, the numbers tie up regardless of which way we do it. But it is much harder though to find patterns and outliers using code. The graph is undoubtedly more useful in this regard.

In the next post we will look into applying some analysis of variance (ANOVA) to examine if the difference in weights across feed types is really significant to arrive at any conclusion on nature of the feed. Thanks for reading!

 

Confidence Intervals for a proportion – using R

What is the difference between reading numbers as they are presented, and interpreting them in a mature, deeper way? One way perhaps to look at the latter is what statisticians call ‘confidence interval’.

Suppose I look at a sampling of 100 americans who are asked if they approve of the job the supreme court is doing. Let us say for simplicity’s sake that the only two answers possible are yes or no. Out of 100, say 40% say yes. As an ordinary person, you would think 40% of people just approve. But a deeper answer would be – the true proportion of americans who approve of the job the supreme court is doing is between x% and y%.

How confident I am that it is?  About z%. (the common math used is 95%).  That is an answer that is more reflective of the uncertainty related to questioning people and taking the answers to be what is truly reflective of an opinion. The x and y values make up what is called a ‘confidence interval’.

Let us illustrate this with an example.

From this article  – out of a random sampling of 1089 people, 41% approved of the job the supreme court was doing. To construct the confidence interval, the first step is to determine if this sampling satisfies the needs for a normal distribution.
Step 1: Is the data from a normal distribution?
When we do not have the entire dataset with us, we use the below two rules to determine this:
1 The sample observations are independent – from the article it seems like random people were selected so this is safe to assume.
2 We need a minimum of 10 successes and 10 failures in the sample. – ie np >=10 and n(1-p) >= 10. This is called ‘success failure condition‘. Our n here is 1089, and p is 0.41. So successes are 1089*41 = 446.49 and failures are 642.5. Both are larger than 10, so we are good.
Step 2: Calculate standard error  or standard deviation of the confidence interval is calculated as square root of p(1-p)/n. In this case it is square root of 0.41*0.59/1089 which is 0.0149.
Step 3: Find the right critical value to use – we want a 95% confidence in our estimates, so the critical value recommended for this is 1.96.
Step 4: Calculate confidence interval – Now we have all we need to calculate confidence interval. The formula to use is point estimate +- (critical value x standard error) which is 0.41 + (1.96*0.0149)  = 0.4392, and 0.41 – (1.96*0.0149) = 0.3807.
So, we can say with 95% confidence that the true proportion of americans who approve of the supreme court is between 38.07% and 43.92%.

We can spare ourselves all this trouble by using a simple function in R as below, and we get the same results. We need to pass to this function what is 41% of 1089, which is 446.49.

Just typing prop.test(446.49, 1089) gets answers as below:

pic

So, with just two figures – the sample count and percentage value, we were able to derive a deeper conclusion of what this data might mean. Thanks for reading!

 

 

 

 

What is networking, really?

I am still trying to get up to speed on blogging after a gap. Today I managed to push myself to write some R code and test it, and it worked. Am getting there, although need more work to turn it into a blog post. So, here is another on the lines of professional development. It is about that word that many people hear of and know of, but really don’t know exactly what it means. I certainly didn’t.
When I was new to the community, I heard many people say networking is the best way to find work. But I really didn’t know what they meant. I thought you had to know a lot of very influential people, and am not the kind of person to seek out people of power/importance and push my cause with them. After a few years, that definition changed. Now I thought I need to tell people am looking for work and they would in turn respond if they knew of an opportunity that was of interest. This is true, but true only partially. It rarely happened. I started telling people I was looking for work, and almost nobody sent me any contacts or information. I was hurt and disappointed when they didn’t. Many times I started considering if it was worth my time to go to conferences/sql saturdays and so on.  It took me close to 15 years to figure out what networking really is, and to get it work for me (at times, it doesn’t work all the time, nothing really works all the time 🙂

To me it is as below:

1 Networking is really just making friends. Get friendly, learn to relax, introduce yourself to new people. Don’t go with any heavy objective or intent. Say you are so-and-so, pleased to meet you and then see how it goes. The next time you see that person, he/she may recall who you are. And there maybe someone else with them that they may introduce to you. That is how the friends circle/network grows.
2 Talk of things you are comfortable talking about. There are many things people talk of that one cannot participate in because one does not share that common interest or simply one does not like it. To me specific topics like that include religion, politics and at times cultural differences. I stick to things am comfortable with and usually find things to talk of in that area.
3 Make your work known via blog posts, talking at user groups or other events. This is by far the most important key to people recommending you for jobs or even letting you know of open positions. If they don’t know what you are good at they can’t relate you to any position you’d be good at. That is partly why I personally didn’t get anyone to recommend me, and I never realized it. Once I got active with blogging and speaking, things changed rather dramatically.
4 Give things time. Networking and building your network takes a long time. Sometimes we can find instant chemistry/connections in people, and the person you talk to today may be your boss or colleague at the next job. But such miracles are rare. Most of the time, people take time to know you and over time that can mature into an opportunity, or a referral.
5 Get active on social media. Many of the friends I have in the sql community are people I got to know better via twitter. I am personally not hugely active myself, but I do read what they have to say and respond appropriately when I can. I also share what I blog and get comments or feedback on that from time to time. Twitter is by far the easiest media there is to make new friends, particularly in the sql community.  It is not as personal as facebook and not as opaque as linked in – it is somewhere in between and is easy to use.

I hope this helps anyone getting started newly with community and networking. Networking is worth it – not only will you gain a lot of support and friends, you will find job openings and opportunities that never existed, guides/mentors who can help you and friendships that can last a lifetime! Best of luck!!