photo credit: FeatheredTar
I’ve almost talked myself out of this post. It’s a lot of self-disclosure. However, I’m going to try and put on my big-boy pants and realize this might be useful to other people.
A lot of us build up a persona. We want to be thought of as smart, competent professionals. We do a lot to support this: our blog posts, open-source contributions, public speaking, participation in local users groups. Not all of this has ulterior motives. It’s a lot of fun to build worthwhile things, to collaborate with smart people.
But let’s be clear. The smartest of us have dumb days. The more I get to know highly-respected people, the more I see them as human, like me. They are still laudable, but they have their quirks, their weaknesses and blind spots.
So do I.
For me, sometimes things build up. The stress builds up, I can’t sleep, I can no longer concentrate. The last thing I want is to hash it out in the open, let people see me weak. I hole up and try to write code. I have a pattern of this, unfortunately. Things build to a head, then I want to drop off the grid and just be in a quiet space for a while. I still write code and think, but I’m a shell of myself in those times.
Maybe this is common. Maybe some of you have experienced this tendency? I’ve seen it in others.
Some would say that this is evidence that I lack a solid foundation. I might agree. The more I learn from Mastery, the more I respect the immersive practice we all need to be competent.
However, the cold reality is that any of us can get overwhelmed in software development. Aspects of it can grind us down.
Sometimes it’s the repetitively inane concerns of our users. On my good days, I know they are real concerns, something I can help them with. On my bad days, I get annoyed with how flippant they can be about work that can be terribly taxing and challenging.
Sometimes it’s the co-workers. It’s easy to take offense at others’ failures. The closer we get to others, the easier it is to get bristly with others. I’ve found that this gets harder and harder the more ‘competent’ my co-workers are. I.e., the really big players tend to have more quirks than the middle-of-the-road developers. Middle-of-the-roaders don’t stick out, don’t risk being personally offensive. The really good ones pick up prima donna habits. But, they also have a lot to offer. They take risks to give more, and they should be respected for it.
Whatever grinds me down, the truth is I can get ground down. There is a lot of good advice around to handle that. Don’t be a hero is one really-important piece of advice. Stop after 8 hours. Sleep. Take interest in life, leave work to only a part of life.
I saw this from @tranqy today, it stopped me in my tracks:
Secret software sauce. Take work in the smallest size possible, communicate early and often with stakeholders, be honest. Everyone wins.
And that’s the reason I told you about my faults. When life seems too big, the code too overwhelming, when I don’t want to talk to anyone (including stakeholders), when I lie to myself about my state of readiness…this is when I lose. We can all get ourselves down. The trick is to realize it and turn it around. Take a break, then a breath, then break it down. As small as possible. I can write a single failing test when I’m down. I can fix a mis-spelled word in documentation when I’m really overwhelmed. I can be honest and write a note to myself about what’s really going on.
If I break it down to something really simple, I don’t have to be a hero, I can get back in the game. Sooner than later.
photo credit: Lauren Manning
The trouble with Seven Databases in Seven Weeks is that it’s a bit of a playground. Ostensibly, it’s about becoming a well-rounded developer. That probably works, if you push through this in seven weeks. I’ve been picking it up more for fun than for progress. I don’t think that’s a bad thing.
The book is well organized. You’ve got seven databases to work on. You get to work on relational, key-value and graph databases. You get to work with systems that do a few things really well and some that are general purposed. You can crank the scale up with some of these, working in large clusters if you’d like. You work from a very basic introduction to beginning to explore and branch out. The topics interweave so that you don’t really feel lost.
There are some pretty hip databases in the list: PostgreSQL, Riak, HBase, MongoDB, CouchDB, Neo4J and Redis. PostgreSQL is becoming the new black (in my circles, anyway). Redis is my favorite go-to for NoSQL issues. But what about the others? Ha! Exactly! You’ll get into these technologies deep enough to not only see these technologies, but to see them in action.
And, that’s the big gap, isn’t it? Getting from the manual to a practical understanding of a technology. So, we write our hello world applications, and we Google around to find tutorials. But then we’ve got to leap out and start experimenting on our own. We’re groping for the big picture. That’s what this book is about.
You’ll learn about technologies surrounding these databases. HBase comes with a discussion of Thrift in a distributed cluster. Neo4J comes with a discussion of Gremlin. For PostgreSQL, you’ll learn about full-text searching and hypercubes. Redis comes with a practical discussion of configuration tradeoffs.
You don’t have to know what any of these are about. You are introduced to interesting things to do and interesting ways to solve them. Mostly, you’re encouraged to roll up your sleeves and get comfortable using these tools. And, if you’re like me, you’re going to find yourself straying from the track. I think that’s a good thing. Competence comes from practice.
Bottom Line: if you’re interested in data, pick up this book. If you want to get a job with these technologies, plan on going much further than the book. All in all, this book doesn’t sit on my bookshelf, it sits on my desk, for every time I want to try something interesting.
I have been bugged by the TV commercials 5-Hour Energy has been running during the Olympics. I was first irked by how many factors they threw into a 30-second commercial. I was then irked by how much fine print they slapped into that commercial. They are suggesting their product is healthy, even approved by doctors.
To my eyes, it seems not unlike cigarette commercials showing doctor recommendations.
Here’s what they’re saying, in a nutshell:
- 5,000 doctors were approached in person
- 2,500 of those responded
- 73% would recommend a low-calorie energy drink if the patient is already using energy drinks and is healthy
- 56% would recommend 5-Hour Energy for those patients that already use energy drinks and are healthy
Of course, they do it with a professional-looking woman rattling off numbers next to about 15 reams of paper. The impression is a lie.
As far as I know, no doctors are saying to use energy drinks. That’s the first issue. Everything doctors actually recommend seems to be to run away from those as fast as you can. But, if you’re going to do this kind of thing, and you are healthy, you might as well take a low-calorie approach.
What if I assume the 2,500 doctors who wouldn’t participate in this kind of study would not recommend any energy drink under these conditions? Then we’re saying that about 28% of doctors would recommend using 5-hour Energy to those that are using energy drinks and are healthy. So, if they’re not recommending low-calorie energy drinks, are they recommending high-calorie energy drinks? Not likely.
Do I have to point this out? Doctors don’t want you using these things. 5-Hour Energy is the snake oil vendor of today.
Ranting about this is kind of silly. Here I am, acting a generation older than I am, yelling at the TV. Sheesh! It’s just grimy business, and they invaded my living room.
OK, I’m going to confess something here. I’m terribly envious of Leigh Dodds. Maybe you have people like this in your world? Bright, effective and young? The guy has done some amazing things, and done them gracefully. Privately, I’ve referred to the ever-nagging problem of keeping sharp as the Leigh Dodds Problem. Leigh is a concrete example of someone that makes me look like a doddering old fool before my time.
I’ve watched Leigh for a while. I watched the rise of his company, Kasabi. I was really sad to hear that he’s closing down. I think Leigh was ahead of us a bit. He built services that I wanted to integrate into several upcoming applications, but the development lifecycle can be a bit slow sometimes (especially if I’m driving it, right?).
Leigh’s work always seems to cut to the nub. This last weekend, I used his Linked Data Design Patterns to help shape some models in a semantic knowledge base I’ve been building. The result was an escape from the academic, practical and rich data relations that weren’t complicated to build. It’s not surprising that Semantic University wanted to highlight this resource today.
Anyway, it feels good to get that off my chest. It’s important that people know they’re making an impact in this world. I’ll keep watching Leigh and expect many great things from him.
photo credit: Lord Jim
When you were a kid, did you like showing off to your mom? Look, I can ride a bike! Look what I made you! Look, a bug!!
Once I took some extra housing wire, stripped the insulation off of the copper and made a ring out of it. I then wrapped the top part of it with a colorful pipe cleaner and gave it to her for her birthday. She loved it. She kept it in her jewelry box. I was so proud. I came up with that idea on my own. Well, my older brother might have helped. I don’t remember that, but he usually did back then.
When I did that, I wasn’t comparing myself to jewelers. They had gold and silver and jewels and training, I was just a kid. I was having fun. I was learning something. It worked. That was enough.
It turns out that this way of doing things is, like the best. (I’m channeling a little sister there, when she was a teenager.) Having fun, being curious, giving ourselves permission to make mistakes is how we get it done.
Somewhere I heard a term for spike, referring to writing throw-away software. I don’t remember where it comes from, but I think about a football player spiking the ball after a touch down. He’s done something great, spike! That’s done, let’s move on, right? So, I wrote a little script that creates a directory, writes a test file, an application file, a test helper, and a readme file. Just a few basic tools for me to try and do something, then throw it away after I’ve learned from the experience.
Since I wrote that little script (here it is, if anyone wants to try it), I’ve had a lot of fun. I’ve learned more. I’ve turned a couple of these spikes into larger libraries that solve real problems. I started by just wanting to learn something by practicing.
And that, my fine friends, is a great way to live.
Learn more, do more, find success, get smarter, don’t wring your hands in anxiety, don’t put off things that will be really rewarding. That’s the whole point of putting getdatachops.com together. It’s a project I’ve been working on to teach data skills to application developers.
You want to build something? Great! Let’s do one together. I’ll show you some things. I’ll give you tips and tricks. You’ll have my notes when you want to do it again (lots of them). Most importantly, you came and did this while your mind was free to wander, while you were interested and could play a little. You did this when your mind was at its optimal place for learning and getting really good at something.
Whatever you do, have fun. It’s good for you.
photo credit: DarkAura
The other day my ex-wife was joking with me about why I choose to write code. At the end of our marriage, I was just learning Rails. It was 2005 and I was overwhelmed with too much responsibility and too little ability. I was mad, crazed even, trying to resolve issues that had overcome me. I was constantly swearing, constantly pacing. From my ex-wife’s perspective, writing software was a delirious process. From my perspective, it was one more stress in a monumentally stressful time.
A lot has happened since then. We’ve made peace, my ex and I.
I’m also making peace with the things I build. This is harder than I thought it would be. The following is a letter to anyone starting out, anyone learning a new skill, someone who has a dream and a long road in front of them.
So, you’ve decided to build things. Good for you. I’ve always been happiest when I’ve been making things. Technology is in the mix? Why don’t you grab a cold beverage and sit with me a while?
I’ve built some things. Sometimes I worked for smart people that helped me figure out what I need to do. Sometimes it’s been all up to me, rise or fail. How do I do what I do? It seems to help to have a loose grip on what I’m creating. By that, I mean, what benefit do I expect to offer when I’m finished? I’m better off having a clear idea about eventual benefits than exactly what it is I’m building.
That’s because things change. Technologies change, your understanding of the thing changes. That’s a good thing. A lot of the frustration comes when something has to be just so, even when it doesn’t.
There’s a story from about 5 years ago. I liked the story because it was a Java vs Ruby story (and Ruby won). The Java team had a deadline they were going to miss. Despite a lot of hard work, something had to be done. A Ruby team was brought in. I don’t remember the full details, but the final solution was a class and a YAML file. The Java team cried foul. The Ruby team claimed that the requirements were met and that the end users were able to have working software. The story’s not really about Java vs Ruby. This is about a fixed mindset vs. an agile mindset. You can do this with Java or C or anything. You just have to be flexible.
In the recent Agile Roots conference, Linda Rising gave a great talk about the Agile Mindset. In it, she points out that a mindset determines goals, reactions to failure, belief about effort and strategies and attitudes about others’ successes. More particularly, an agile mindset sees ability as something that can grow, like a muscle. The goal is to learn, rather than to look good. Challenges are embraced and failures are learning opportunities. Effort is part of the path to mastery.
photo credit: Renaud Camus
This is different than swearing and kicking chairs. Do you see that? Kent Beck reminds us that perfect is a verb. This is a process, we are here. Where is here? I’m glad you asked.
Andy Hunt reminded readers of Pragmatic Publishing books about how our minds work in Pragmatic Thinking and Learning. A core piece of that book was learning to groove with the Dreyfus model. Start as a novice and find recipes and tutorials. Change the recipes just a little and you’re an advanced beginner. Continue to experiment until you can see the big picture. Now you’re competent. Good for you, there’s still lots to learn. When you can self-correct quickly, you’re even proficient, but you won’t be expert until you really get the edge cases and people are seeking your advice.
Few of us become truly expert at anything. There’s just too much to do. We keep learning and sharing and failing and not failing. Along the way, if we can keep our wits about us some of the time, we’re making progress. And that’s something. That’s enough to get a job, finish a project, become a consultant, solve a problem.
photo credit: borispumps
And if your particular goal is to use Rails to build web applications, you should know something first. If you don’t know Rails, you need to know that Rails is kind of growing up. In the beginning, it was so much fun. Everyone was learning. A lot of us picked up Rails and then had to catch up with Ruby. There were cookbooks and cheat sheets and lots and lots of good articles springing up all over the place helping us figure out all the little quirks. That’s because, as a community, we were all novices then all advanced beginners. So we were intent on making things easier for the new guy. This is the age when RailsCasts, Peepcode, and Twitter were born.
The community is still very willing to help, but there are more and more people intent on becoming expert. Their focus is on understanding the idiosyncrasies of their now-complicated and demanding applications. Twitter grew up from a couple of tables in a database. So have all of our applications. I’m just saying, the Rails community is noisy, often about fairly advanced topics. Don’t worry, stick with the basics, find a mentor, practice.
It’s the same for anything, right? We’re figuring it out.
I recently read a blog post about the symbiotic relationship we have with technology. It made me think a little about what we’re really after. I mean, if the computer can search and sort and filter for me and allow me to be somehow more human, well that’s something. Building these kinds of things for other people, well that’s something else. Knowing that you are here to forge that relationship might bring to mind the idea of heat and pressure and direction. Technology has to become something you can eventually push around. So push. (meaning, build, do, practice, get comfortable with the sometimes uncomfortable)
photo credit: Steve & Jemma Copley
If you’re going to build with computers, you’re going to get to know their personalities. You’re going to write a query that takes too long, a bug that’s hard to find, a layout that just doesn’t match up You’re going to hit hard details, the cogs of the wheel are grinding you down like teeth in a ravenous animal. My advice? Drop a few expletives, kick something, stand up and walk away. Take a breath, don’t take it personally. Examine your mindset. It’s OK.
Now what? Get into a dialog with yourself. Seriously. Write down a question and answer it. Drill down to the problem, walk around it a little, tease yourself, call your own bullshit, prioritize, filter, regroup, restart. Language is a tool that takes you somewhere else. Did you know that? Your mind actually changes state by talking. With a piece of code, you call a method. With your mind, you can yap away. Language uses finite words and sentences and meaning and fires different synapses in your brain and changes the things you see. It changes the scenery, changes the chemistry, changes the tools. It’s like walking out of the blacksmith shop, across the street, into the steel refinery. Now, you’ve got some different tools, and you can see them, and you can see that you have something different you can do about your problem.
Blogging helps too. You don’t have to feel so uncomfortable about schizo tendencies if you’re talking to something other than the voice in your mind. But talk it out, realize that your effort is building a capability in you.
Ra, ra, ra, right? David the cheerleader. Maybe, but here’s the nub: I went through hell until I could do a few things differently:
- Groove with change
- Change my mindset
- Apply learning resources conscientiously
- Start a dialog
The peace comes when we can finally see how our efforts are making things better. We have abundant technology to help us along. The early stages can be daunting. I hope this helps you apply yourself well in the early years of becoming a proficient problem solver.
photo credit: pat00139
I’m collecting success stories. People who have delivered the goods, overcome their fears, learned to live at a happier level. I’m looking at people I respect. I’m looking for people you may know that inspire you. Who do you know that has overcome their self-limiting beliefs? Who has reached for the stars and given their best?
Here is a message from someone I know to his son. This person is quite successful in his life. I’ll keep his name anonymous, since I haven’t asked his permission to use his name here.
How many people are on this earth? 7 billion? 7 billion people competing for each others’ position in life. At times we compete with small groups of them, and the ones who are competing at a worldwide level have already passed you. But here is how you can get to their level and pass them: take advantage of opportunities wherever they come up. Don’t be afraid to fail, that is how we succeed, how we become better people. The people at the top of the world are winning too often, that slows down their rate of learning from their opportunities and being the best. The people who learn the most are the ones who are not afraid of failing and take advantage of their opportunities. Compete to the level where YOU are the one leading, NOT FOLLOWING. I know we are all good enough to be the best at what we do. Don’t be afraid of taking your chances.
Tell me stories of people that inspire you.
photo credit: bgottsab
Jason Patorti asked this today. Lots of thinking ensues. I’m OK with my next product being something simple. It is looking like it will be a knowledge management product. I’m almost to the point where I can say that for sure. Here’s a dialog that went on in my head:
Isn’t this a recipe for a bloated pitch?
How do you mean?
I mean, a startup needs to do 2 things: prove it can focus, and prove it can focus on things people are willing to pay for. Changing the world is, by definition, a bloated pitch, a bloated goal.
Yeah…I don’t think he meant that exactly. He’s not calling for Quixotic dreams.
How do you know?
Well, I guess I don’t, but votizen is simple enough: share how you vote, use basic technologies to connect. This wasn’t tilting at windmills, but it’s a startup with a purpose. People won’t be hurt for civic participation.
Ironic that you bring this up during a West Wing marathon.
Maybe, but the point is, what are you doing and how are you impacting the world?
Well, I haven’t committed myself to my first product yet.
Yeah, what about that. Are you chicken?
Probably. I gave myself permission to work in the lab a little before trying to release a product.
Fine, but I think I can still do some good.
Yeah, how? Also, don’t you think this is rather ironic? I mean, you’re saying don’t pitch unfocused business ideas, but you’re “in the lab.” Isn’t that code for irresponsible, work as long as you want in any direction you want?
Hold on there. Yeah, maybe a little, but I’m making progress.
Progress towards what exactly?
An age of decision.
And you call Jason Quixotic?
I didn’t do that, exactly…whatever. I want to deal with the flood of information so that I can live a grounded, decisive life.
Sounds ambitious, to say the least.
Maybe, but it’s important. Look, I read hundreds of messages, thousands. I’m hearing tons of stuff about how it takes too long to process through the information.
For example, journalists working under deadline. It takes days or weeks to produce graphics, to summarize information in ways they want to provide the information. This is a problem of too much information and too little time to process it all.
And I’m hearing about issues about sketching with data, again from journalists.
Sketching with data?
It was a concept about using the data to outline the story. Understand the data, and you’ll get to the stories that impact lives. More relevance, more integration, that sort of thing.
What’s your obsession with journalists?
These guys are the canary in the mine shaft. They have the biggest problem with handling the flood of information, and their whole industry is in danger because of it.
In danger? Aren’t you being a bit alarmist here?
Kind of, maybe. The thing is, the local papers are closing and consolidating, the micro publications are going out of business, and the big companies are talking a lot about the changes they’re making.
Well, they’ve coined the phrase data journalist. They’ve been talking about it for a couple of years now. They just put out the Data Journalism Handbook. Publications like the New York Times and the Guardian are advocating data journalism, showing how they’ve been doing this all along. The Guardian’s highest traffic is to their data section.
OK, so let’s say you get a knowledge management product together for journalists. Let’s say you’re wildly successful with this. How is the world different?
Well, hopefully we get a little clearer about what’s important in this world. We hear less news reports about Tweets and more reports that are based in substantive research.
That sounds like it’s an ambitious first product. Are you sure you’re ready to take on something quite so big?
I’m not, actually. That’s why I’m taking extra time. I’m trying to get down to the heart of things, the essence of a grounded, decisive life, and build the bare minimum for that.
OK, let me hear your pitch.
Drowning in too much information? You can’t keep up with all the Tweets and blog articles and everything else to read? Use Archie. Live a grounded life. Put things in context. Take life a little at a time, but feel like you’re keeping pace with the important things in life.
Wow, is that the final pitch?
Definitely not. It’s a start though.
How are you going to do that exactly?
Ah, I’ll talk about that another day?
You know how?
I’ve been doing it. It’s working, “in the lab”, as you say.
Fine, I’ll be excited to see what you’ve come up with.
Hopefully others will be excited too.
And this is for journalists?
No, that’s for later. They need more than I can give right now. This thing I’m doing, Archie, it’s for thinkers. It’s for people who make a living by keeping up with a lot of information. Developers, consultants, maybe journalists too, but generally. The good stuff, the really amazing, focused stuff that I want to give journalists, that will take longer.
Well, sounds good.
Yeah, I think so.
So, there you go. A little preview of some things I’m working on. My take on starting up with a purposeful beginning. There’s more too it, of course. I wrote recently about my experiences with 30×500 which teaches this process.
photo credit: Big D2112
When I first heard about sabbaticals, I wanted in. I was young, probably in high school or even junior high. The idea that someone would get paid to do whatever they wanted sounded about right to me. Imaginative thinking is the best kind of thinking. In my mind, I pictured bright thought leaders taking time to do brilliant things. I wanted to grow up to be that guy.
This TED talk by Stefan Sagmeister is about this more traditional approach to sabbaticals.
Unplanned mini sabbaticals come up for consulting. Consulting has its highs and lows. For technical consultants, if things are going right, a few things take place:
- We start with a passion in a particular area
- We take the time to work up an expertise in that area
- We get hired to participate on a project
- We leave and take on other projects (sometimes with breaks between engagements)
This can be a great win for everyone involved. The company can get quick results in areas where they are not yet expert. The consultant is passionate, sharp and effective. Between engagements, the consultant can do personal projects, mini sabbaticals.
I’m coming into a mini sabbatical. I’m finishing a project, and several other projects are beginning to line up. I have time to think and explore. I ‘m having a blast: R, D3, Data Visualization, Object Oriented Rails, Semantic technologies…this is a good life. It comes with plenty of ups and downs, but I’d rather be a passionate thinker with some instability than a drone stuck somewhere without an outlet for my curiosity.
photo credit: Maxwell Hamilton
The nub: for people thinking about filtering data, power tags may be a concept to show distinction.
I’ve been thinking a lot about information relevance. I think I’ve written 50 pages of irrelevant essays on the topic (ironic, huh?). In order for something to be relevant, it could have a lot of qualities: timely, rational, focused, conclusive, comprehensive, etc. One particular quality that I’ve been exploring is distinction. What stands out? Which are the outliers? Why is it different?
Tags can give us a clue about distinction. Traditionally, a tag is a folksonomy: it’s loose, open-ended, and encourages some level of commitment to the content. This isn’t enough. I came up with the idea of a power tag, lacking a better term for the concept. What I mean by this is I:
- constrain which tags are available
- break out the thesaurus and gather precise words
- order and scale these words
Let me give you some examples. In the space of intelligence, there is creativity. Some words that are related to creativity are clever, imaginative, inventive, original and ingenious. I’ve ordered these words, and scaled them. That is, if clever is 0, imaginative might be 1, inventive 2 but original 4 and ingenious 6. That is, there is a bigger gap between inventive and original than imaginitive and inventive. I want to have some sort of an idea of how much of a thing is expressed by the power tag. Does that make sense?
Put another way, I’m looking for a way to show distinction with nothing more than a simple tag. I want my tags to say more than just something was cool, and gather all of the cool things. I want to know if something was uber cool (whatever that is, I’m too old and square to know these things any more). Remember: distinction, outliers.
Something can be relevant because it stands above the rest. It deserves attention.
Here’s another example. Let’s say I was collecting a mastermind group (I am, by the way). That is, a group of people that meet regularly and push each other to meet their goals. Effective mastermind groups have highly motivated and skillful people in them. If my goal was to create a group full of entrepreneurs launching their web applications, I might look for people with technical skills, gumption and critical thinking. Each of these could be a category for power tags. So, under technical skills, maybe competent, proficient and expert (I know, technical skills is a broad category, bear with me). For gumption: clod, productive and prolific. Critical thinking? bright, penetrating and enlightened. If I mean it when I call someone expert or a clod, then I can start to distinguish people to invite to the group.
And if I can’t find an enlightened prolific expert? Then I organize a group that balances itself. I may get a clod, someone who hardly ships anything, but that’s because they’re a perfectionist. If I mixed them with someone who is competent and prolific, they should be able to inspire each other to greater heights. See how this could work? The relevant people could pop out of a pile and stand out.
Now, why would I spend my time thinking about this? Information overload. Research for future products. Managing my own knowledge bases. A desire to understand information better, to filter, to get to that place where decisions can be made. Actionable insight comes after filtering the deluge of information. It’s been coming over the sides, my boat has been ready to collapse, drowning by information could be my cause of death if I’m not careful. I’ve been putting these into my own knowledge base, and so far so good.