Working on the Marketplace API

Published in Open Source

The final week of the semester before exams is now coming to a close and in between studying I have been working on a couple of different bugs on the Marketplace relating to its API. The Marketplace is not a 100% API driven application (yet) and the bugs that I took on this week should help make the API more robust and bring it closer to a fully API driven state with added functionality.

Preventing Nameless Collections

Earlier in the week, I found that there weren’t a whole lot of new interesting bugs on Bugzilla, which is atypical (although it likely does signify a good thing) so I reached out to one of the full-time Marketplace developers on IRC. He ended up coming back with a whole bunch of great bugs which were all tasks that were assigned to him that he hasn’t had the time to get around to yet. The first one that he suggested described that a new collection was able to be saved via its API with an empty name. This, of course, should not be possible so I set out to discover why this was happening.

I started up my development server and navigated to the Curation page in order to create a new collection. Once this was complete, I tried to remove the name of the collection but I seemed to be restricted by some front-end validation that was making the field required and disabling the submit button. I dug through the client code in my browser and I removed the flag that was making the input box required. I also changed the submit button from a disabled state to an enabled one. Upon doing that, I found that I was able to save a collection without a name, exactly as the bug suggested. It seemed that we were relying on the front-end for validation and not validating on the API side, which is not a great practice.

I started to look through the API code and I found where the collection was being updated. I had to brief myself on the Django Rest Framework documentation surrounding ModelViewSets since we seemed to just be using the basic built in update functionality of the framework. I noticed that the field on the Django model being used was a PurifiedField, which is used to strip out html for translations. I ideally wanted to find a way to modify the field at the model level so that if all locales were empty, a validation error would be raised. I tried to modify the translation fields being used for a while but nothing seemed to be working properly so I opted to go to a higher level to see if I could solve the problem in a different way.

I began reading about Django Rest Framework Serializers, which are responsible for taking the incoming JSON data of an API request and translating it into Python/Django objects so that they can be utilized by the back-end (among other things). Through my reading, I learned that it is possible to add a custom validation method on a serializer, which seemed like it could be a good solution. In my validation method, I opted to loop through all of the locales that were submitted in the request and if all of them were empty it would raise a validation error (meaning the data is not saved). This seemed to work well from my manual testing so I added a couple of unit tests that would verify this behaviour. The tests would ensure that a validation error would be raised for the case where the submitted name was a simple string or for the case where the name was a dictionary of different translation locales. After this was complete, I opened a new pull request for my fix on the Zamboni project.

Creating An App Icon Update API

With my previous bug in good shape, I started to explore a new bug which was to create a new API endpoint for updating an app’s icon. Since I was dealing with app icons, I decided that it would first be a good idea to get icon loading actually working in my development environment so that I would be able to visually see the affect of my new API. This is something that I was neglecting for a long time since I figured it would be a difficult task but as it turns out it was quite easy to enable with a few lines of code in my settings file:

SERVE_TMP_PATH = True

ADDON_ICON_URL = '/tmp/uploads/addon_icons/%s/%s-%s.png?modified=%d'

Since the intention was to update the existing app icon (all new applications already have an icon specified), I opted to make the endpoint respond to a PUT request instead of a PATCH request, which would be more appropriate for an incremental update. The nice thing about working on an established project is that 9 times out of 10 someone has already done something similar to what you are trying to implement. I often find myself scouring the repository for examples before I dive into implementation.

I ended up adding a new method on the AppViewSet, which would accept base 64 encoded icon data and an icon type (JPG or PNG). I created a new form that would take in the provided JSON data and convert it into the format that is expected by the existing Django form that is used to update an icon. This was great since it meant I got to take advantage of all of the existing icon validation code. If all validation completes successfully, I chose to simply return a 200 OK response and if not, an error message will be returned with a 400 BAD REQUEST error.

Once I was happy with what I had written, I had to figure out the best way to add tests for this. I ended up creating a new test suite that currently consists of 5 different tests to ensure the basic functionality. One of the tests was to ensure that CORS was working properly for the PUT HTTP method. I added another to ensure that the icon being submitted and the app’s new icon were equivalent. One of the tests ensured that the API response was 200 OK upon a successful update. The final two tests were to ensure that people without app ownership would be denied from updating an app’s icon and to ensure that only PNG or JPG icons could be uploaded.

I added a new section in the documentation to detail the new endpoint that I had created and I opened a new pull request here. I have not received any feedback on it at the time of writing but I expect that to change pretty quickly!

I’m thrilled to actually be able to make these incrementally larger and larger contributions to the project over the course of my time working on the Marketplace. The practical applications of the work that I am doing are quite substantial and that is a great feeling!


Written By

Andrew Halligan

Published April 4, 2014


Related Articles