This week I’ve been working on the first project of Udacity’s Self Driving Car Engineer Nanodegree – annotating lane lines in images and videos recorded from a front-facing camera mounted on the front of a car.

There are plenty of blog posts floating around explaining the concepts (here’s my project code with a walkthrough of the process of annotating images), so I thought I’d go through some of my bugs instead. Here’s to what happens in real life.

**Bug 1: Detecting line segments**

Yikes! What’s that mess at the top?

(What’s going on? The white lines are tracing the points where the change in colour is large, e.g. from white to grey with the lane lines.)

I only want to detect lane lines, not trace out cars too. I restricted the region of interest (this is the area the algorithm cares about – it throws out all points that aren’t in that trapezium) further downwards and it was fine.

**Bug 2: Extrapolating Lines**

￼Tradeoffs:

Hey, my line is downstairs, not up there!

What’s going on? To extrapolate each line, I ask the algorithm to draw me a line based on my calculated slope (steepness) and intercept (what point the line goes through to locate the line, specifically the y-value when x=0). So I ask it to draw a line between points (a, calculated_value[a]), (b, calculated_value[b]).

Turns out I’d misspecified one of these points by getting the x and y coordinates (height and width) mixed up. 😛 Usually in maths the left-right coordinate goes first, but here the up-down coordinate goes first. Second problem solved!

**Bug 3: Peskier lines**

The line on the left is just wrong.

Here I want to share what I did to investigate the bug. I printed the coordinates of the points my algorithm used to extrapolate the linens and plotted them separately. This was to check whether the problem was in the points or in the way the points were extrapolated into a line. E.g.: did I just throw away many of the useful points because they didn’t pass my test? Turns out the points detected included outliers – you can see an anomalous point in the top right hand corner.

**Bug 4:**

What are you doing? There’s no lane line there!

It turned out I wasn’t capturing enough of the right lane in my region of interest. Remember – I’m throwing away ALL the points that aren’t in my region of interest. So if the right lane isn’t in my region of interest, I’m not getting any of it.

The algorithm probably then took some random white point in the bottom left and tried to draw a line from that. Just another day at algo work.

**Bug 5: NaaaaN – Extrapolating lines when none pass our test**

Remember how I asked the algorithm to draw a line between two points? Well it’s telling me that one of my points is not a point because its coordinate is NaN. Not A Number.

Easy fix: Don’t require them to pass your test. I had required each line segment to be at least 50 pixels long to include their endpoints in my calculate-my-line-from-these-points algorithm. Turns out in this case no line segments were longer than 50 pixels. So I added a clause underneath that said ‘If there are no points recorded, go through the line segments that exist (with e.g. negative slope) and add all of their endpoints to our list’. That solved it.

It’s a buggy life! Do tell me what you think about this type of post in the comments. 🙂