Testing: The Art & Science Asking Questions
Why are we so keen to ‘get the answer’ and not ‘ask the question’?
I’ve been told, for most of my learning life, that I will be rewarded with gold stars and good jobs if I can answer questions.
All my life I have learnt to pass tests; spelling tests, maths tests, GCSEs, A-Levels then finals at Uni.
Life is geared towards ‘getting the answer’ – few of us think to ask whether the question being asked is the right question to ask and hence worth answering.
There are no gold stars awarded for opening your exam paper and writing 10 alternative new questions then answering those instead.
In fact, you’d probably get a big fat FAIL.
And so, whilst we’re all pretty good at deploying our knowledge to answer questions, most of us are shit at asking good questions. We kind of skip to the ‘important’ bit where you answer the question and get the gold star.
However, the art of asking the right questions (a.k.a “Testing”) is just as skilful, if not more, than answering questions. As these exam answers show – there’s a lot of room for misinterpretation.
There’s even more room for misinterpretation when you’re testing a computer.
Coding is a bit like trying to direct a blindfolded person with a very very limited vocabulary, who is standing in a different room to where you are standing, to a sit on a chair that neither of you can see. You’re trying to speak to them in a language you don’t really understand and all you have is a vague map of the room next door, but you don’t even know if you’re holding the map the right way up and you might be having trouble understanding what the symbols on the map mean.
This is why you need tests.
To test if you’ve got the map the right way up.
To test if your blindfolded person heard you, then understood you when you asked them to move right.
To check they actually moved right.
To check didn’t walk into the wall in the process of moving right (if they had you might have had the map the wrong way up…).
To check that, in the end, the person ends up sitting on a chair and not a beanbag.
It’s kinda like something from the Crystal Maze. Wow, what a show. I hope it comes back soon.
Coders are Testaholics.
They have entire theories and concrete working practices for testing your code to find out if it’s doing the right thing.
They have banter about testing code.
They have programs with mental names that you can use to test your code.
(I took the screen shot above from here – honestly, this is what it said. It made me hungry until I remembered that I don’t like rabbit food, which cucumbers, gherkins and spinach undoubtedly are.)
They get high off green lettering.
Red lettering negatively impacts their mood.
When you learn to code, developers’ intense focus on testing can be super counter-intuitive.
Like, really fucking odd.
Not only have you been taught for most of your learning life that getting the answer is more important than asking the right questions, but also, when you first start coding you write code in a way that is totally unaware of testing.
You’ll probably have more flow, writing one-liners of method piled onto method that get you to your answer. Stuff like this:
string.split("").each {|x| x*2}.join(" ") Writing code will make you feel like you’re being a bit of a whizz when it passes when you run your program from the command line to…
… test it.
So, actually, you have been testing stuff already, you just didn’t realise it.
When you run your program from the command line, you are actually testing whether it is ‘behaving itself’. You know, playing ball…
BDD (Behaviour Driven Development)
This is when you test that your program behaves in the way it is expected to. This is the ‘above the hood’ stuff – like when you flick a light switch, the light turns on.
Tests that do this are called ‘Feature Tests’ or ‘Acceptance Tests’. They’re kinda results-driven. A sort of, “if I put in x will I get out y?” scenario – they’re less concerned with how x became y. Some nutters might call it testing the “outside” of the program, because you’re not testing the stuff ‘under the hood’, but just the stuff that a user can interact with.
I’m not convinced programs can have outsides when they exist inside computers.
When you start out, you’ll probably run these through IRB (Interactive RuBy shell – this is a ‘REPL’, Read Evaluate, Print and Loop ThingyMcbobbin that allows you to run your programming script more than once and feed in different commands.)
What you’re probably less familiar with is…
TDD (Test Driven Development)
This is when you test all the little tiny weeny steps that need to take place to get the program to behave in the right way. This is all the ‘under the hood’ stuff. This is like checking to see if the lightbulb’s filament has gone, or the wire coming out of the lightbulb hasn’t been chewed by the cat, or that the switch has been wired up correctly.
Tests that do this are called ‘unit’ tests. They’re kinda process-driven. They’re more interested in breaking down everything into tiny weeny steps to see if the steps were carried out as expected. They’re more interested in the route between x and y.
The point of this is that you can pinpoint exactly where things have gone wrong. It’s about identifying the weak link in the computer’s chain of command. It’s apparently super helpful to real developers and saves a lot of time when it comes to fixing bugs later on.
Imagine if a customer came to you and said “the lightbulb isn’t turning on” and you had no idea about how lightbulbs worked, or electricity worked, or how a switch worked. You wouldn’t be able to sort it out at all and would probably just end up bashing away at the switch angrily.
Unit testing is the most counterintuitive of all testing.
Why?
- Because it’s so slow. So painfully slow. So unbelievably myopic and time consuming. Every little goddam thing needs to be tested. EVERYTHING. For somebody with ADHD, this is so freaking painful. LET ME WRITE PROPER CODE ALREADY! It’s forensic, it’s iterative, and it can be really frustrating because…
- If you don’t know what your code is doing in absolute detail, you don’t know how to test it.
- Rspec, the language used to write ‘tests’ that you run your code through, is written in a bastardised form of Ruby that is similar enough to lull you into a false sense of security about writing and reading it, but different enough to thoroughly screw with your head. Because of this, you’ll trust the code you’re meant to be testing more than the tests that are meant to make you trust your code more. Which makes you seriously question the whole notion of testing.
(I’ll come back to Rspec another day in more detail as I’m still not 100% confident with it and I haven’t thought of a nifty way to philosophize around it to make it more digestible.)
But, the biggest headfuckery is writing the test first.
You have to write a test then make the code pass it.
You must fight all of your instincts and years of learning how to answer questions and say: NO! NO! I WILL NOT ANSWER THE QUESTION.
I WILL ASK THE RIGHT QUESTION.
Which is problematic, because you’re asking a question in a similar but different language to the language that you’ll use to answer it in your script code.
Ok. Let’s try this:
describe IHateRspec do
it { is_expected.to respond_to(:brainmelt)}
end
You are meant to write the absolute minimal to get the Rspec test to pass. THE MINIMAL.
No racing ahead with the answer. No. You must stop.
class IHateRspec
def braimnelt
puts "It hurts, it hurts"
end
end
Run Rspec from the command line.
RED? WTF?
I know my code is right. All I did was define a method. I didn’t even write the method.
I DIDN’T EVEN WRITE THE METHOD.
You start messing about with your rspec, which you assume is the culprit.
(2 hours pass)
Why can’t there be a test to test your tests?
(2 hours pass, you miss lunch)
Or perhaps your code is wrong.
(10 seconds pass)
Oh.
Typo in my code.
Goddamit.
illogicode 