Exactly 3 months ago I started my Outreachy internship as someone with little understanding of the exact scale of open source projects. Today I’m here before you to tell you that it was beyond anything I could have ever imagined. Over 22 million lines of code and thousands of developers working concurrently across the world, on many different areas of the kernel; its truly amazing that even after decades of its existence and multiple hands in the pot, that the kernel still remains as consistent today as it was 20 years ago.
Update from last post
As you may remember from my last post, I outlined what my mentor and I had proposed as our way to complete the VMBUS tap infrastructure, that would allow us to inject delays and errors into the Hyper-v guest/host path. I submitted my patches after completing this and we got feedback, the major thing was that the community wanted my code to go into a different file system of the kernel. I originally proposed we use sysfs, but we all eventually agreed it was better to use debugfs to serve this purpose instead. This is good because community feedback ensures that the code that is submitted is good code and also the best it could be before going into the kernel tree. However the major thing for me with this is that my old code relied on the preexisting sysfs code currently within the Hyper-v drivers. So what this meant was that I would have to expose the Hyper-v devices to a completely new file system (debugfs in this case). At this point I knew absolutely nothing about debugfs, and I needed a deeper understanding of how Hyper-v device probing worked/exactly at what point in the code flow do Hyper-v devices get created.
For this it took me a week of constantly reading the code across the multiple C files in drivers/hv/ to figure this all out. I also had to read the documentation in https://www.kernel.org/doc/Documentation/filesystems/debugfs.txt so that I knew which API’s i had at my disposal. Thankfully since I already had a design, implementing this was a lot easier than I thought as I just needed to transfer my code that relied on sysfs to use debugfs instead.
Another thing to note is that since debugfs isn’t usually configured into the kernel, this meant that I had to wrap all of my code under an #ifdef and create an entry into the Hyper Kconfig file corresponding to my test code. This allows for the following:
- My configuration in the kernel will only appear once the HYPERV and DEBUGFS modules are installed into the kernel
- When my configuration option appears, my test code itself will only be compiled into the kernel after you have changed my new config option (HYPERV_TESTING) to ‘Y’ on the menuconfig screen. I set the default to ‘N’ so my code will never interfere with regular non-test Hyper-v operations and will not appear in the binary when it is compiled. This prevents the Hyper-v guest itself from running my code when testing is not enabled.
You might be wondering what a Kconfig file is and all this talk about modules and configurations. So, i’ll explain it briefly, all drivers for example will have a Kconfig file that allows you to set a number of configurations to be added for your specific driver. These configurations allow for the kernel to be modular, where if you don’t want something in the kernel after you compile it you can turn it off or vice versa. You can even make something into a module that can be loaded and unloaded while the kernel is still running. As you can probably guess, there is documentation for that as well https://www.kernel.org/doc/Documentation/kbuild/kconfig-language.txt.
After I did this, I tested my new patches against the latest kernel version (using Linus’ tree as my master branch) and confirmed I didn’t have any conflicts in git. After this it was off to test my code and my tool with the new debugfs location in the cloud. Thankfully at the start of my internship Microsoft was kind enough to give me Linux developer access to azure and with that, access to virtual machines with beefy specs. I tested my code and my new user tool for a few days before I finally submitted my code back to the Hyper-v mailing list and to the maintainers. The only thing I didn’t get to add into my code/tool was error injections, but maybe another future intern whose reading this will add it into my code/tool next round :).
- Official Patch to my python tool at: http://bit.ly/36ElcCZ
- Official Patch to C code to tap into the vmbus at: http://bit.ly/36Fz9Ay
I think a major key to my success during my internship was that I didn’t rely on my mentor to tell me how to do things. Don’t get me wrong when I struggled and was stomped I asked for help but only for a specific thing. This is because I read a lot of documentation and I read a lot of code, so that I could build an understanding of what exactly was going on first before trying to ask for help. Something to think about if you’re ever in an internship anywhere is what I like to call “flow”. In order for you to understand a specific piece of code, you need to understand the applications control flow before you get to that piece of code. If you can trace the data flow from start to where that piece of code is called then you will develop a deeper understanding of how that code works, why that code is there and what that code does. After developing this deeper understanding, things start to become a lot clearer and will help you to implement something that uses whatever it is that you now understand. Constant struggle can be good because if you don’t struggle there is no growth, but be wise don’t try to do it all in one go and if you get stuck your mentor and your community are there to help. Always remember to try to develop an understanding of how something works before diving in. Plan and design (even if it seems small) before diving into it.
What’s next for me?
The past 3 months have been great and I now feel like I’m on a good path to overcoming my imposer syndrome when it comes to tech. Now its time for me to head back to university with my head held high and in good spirits about what I was able to accomplish these last 3 months. I don’t know if I explained this already but I’m a junior attending McMaster University for Software Engineering up in Canada. I’ll still be around, so if anyone wants any advice you can email me at firstname.lastname@example.org or message me on Linkedin. Also I’m gonna be at this years Richard Tapia Conference (2019) in San Diego helping to spread the word out about Outreachy, so if you’re planning on attending come say hi to us at the Outreachy booth. I’ll also be at the Open Source Summit Europe Conference (2019) in France to talk about my project, so if you’re there as well you can meet us kernel interns there. Lastly I’d just like to give a big thank you to Sage and the Outreachy crew, the internship was a great experience and I believe many other interns can also express my appreciation and admiration for all the hard work that Outreachy has done and will continue to do in the future.
Till next time…