Reproducing Bugs is Hard

Published in Open Source

This week I am jumping right in and looking to tackle more bugs as I continue learning about the Firefox Marketplace and Add-ons projects. Since my last blog post, I have tried to brush up on any documentation that I can find. This gets a little tricky at times because I am never 100% confident that the documentation is up to date and actually reflecting the current state of the project given that is moves so quickly. While versing myself on the accompanying documentation is important, I am hoping that diving head first into the projects will allow me to ramp up a little faster but only time will tell if this strategy pays dividends.

As I was researching and identifying potential bugs that I could try to solve, I realized that my personal fork of the Zamboni project was a little out of sync with the main repository. Most of my contributions using Git in the past have been on repositories where I was granted access so I didn’t need to worry about this. As it turns out, the steps are quite simple to update your fork to the latest version. First you add a new remote, which in this case has been named upstream.

git remote add upstream

After this is complete you should be able to pull in from the upstream repository to your local copy:

git co master

git pull upstream master
git submodule update --init –recursive

I learned that it is also a good idea to update your submodules as these can change fairly frequently as well. After pushing up to your fork, you should now be in sync with the main project (although probably not for long)!

After that preliminary process was complete, I searched a little more and identified a bug that seemed like it would be good to take on. The description described a situation in the Firefox Marketplace where a rejected application was unable to be deleted by the developer that had uploaded it. You can see the bug here. I was able to reproduce the majority of the steps that should have been required to initiate the bug: I submitted a new hosted application, navigated to the reviewer tools section, rejected the application, and navigated back to the status page of the app. There was one problem at this point; the delete option was glaringly present on the status page! I tried a few more variations to make the bug appear but I finally gave in. Everything seemed to be functioning on my end. I added a comment to the bug on Bugzilla to let them know that I was unable to replicate the bug and asked for some clarification.

There are a few different cases that may make something like this arise. Either I didn’t understand the bug description properly (quite possible) and wasn’t able to make it happen or this particular bug was fixed at some point but the ticket was never updated. In fact, there may be some special circumstances that caused this bug to happen in the production environment that I was simply unaware of. Either way, I was able to easily draw this conclusion – Reproducing bugs is hard!

I continued searching for some other bugs and left some comments on a few looking for clarification such as here and here. I have found that it is quite difficult to find bugs with a sufficient amount of information in them to be able to work on them from the get go. I have been hanging out in Mozilla’s IRC quite a bit (#amo, #marketplace) and have been reaching out to some of the main developers in order to get clarification and ask for any bug recommendations. Based on my previous experiences, this seems like one of the major challenges of a distributed work environment. During my co-op placements, asking questions was as simple as turning my head left or right or rolling my chair a few metres in either direction. Differing time zones also come into play but luckily there is almost always someone who is responsive.

Try, Try Again

At this point I felt that I had exhausted my search for bugs to fix on the Marketplace project so I decided to switch gears and look through the Add-ons project. Within a short while, I came across a bug that described a problem with the collections feature of the Add-ons site. A collection as Mozilla describes it is a “group of related add-ons that anyone can create and share”. The idea is that you could package up a bunch of your favourite add-ons and share your browser experience with others. The main issue is that bots are submitting empty collections that only consist of a link in the description. This type of spam is fairly common but it was causing a lot of headache for some of the reviewers so they wanted to take care of it before it got too out of hand.

After thinking about this problem a little bit, I figured the best way to proceed would be to prevent anyone from posting <a></a> tags in their description and to throw a validation error if they were to have any. I ended up modifying the collection form so that upon cleaning the description field, I used Django’s built in replacetags function to attempt to remove any <a></a> tags that were present. If the length of the cleaned field was less than that of the original field, we would know that someone attempted to place links in their description and thus we could reject the submission and inform them of the problem. I also added a couple unit tests that would validate this behaviour. After I was satisfied with the functionality, I submitted my pull request to the Zamboni project, which can be found here.

A Mix-up

The pull request garnered a lot of discussion immediately and I addressed some of the concerns that were brought up. In that discussion, I learned that someone else had actually opened a pull request to address the exact same bug that I had. We had quite different approaches and interpretations of the problem, which was good but this hinted at another challenge with open source development. Keeping track of what everyone is working on to ensure there is no overlap is a tough problem. The worst-case scenario is that someone ends up wasting time fixing a problem that was already being handled, just as it happened here. At least in my case the learning experience still makes this contribution a win. Hopefully things will run a little smoother next week.

UPDATE: I ended up closing my pull request in favour of the first pull request that was opened. There was a lot of discussion but ultimately the final solution will likely be a hybrid of both pull requests that were drafted. It’s a bit disappointing but I gather that it’s pretty typical in open source development. On to the next bug!

Written By

Andrew Halligan

Published January 24, 2014