NOEL:  Hello and welcome to the Tech Done Right Podcast, Table XI's Podcast about building better software, careers, companies and communities. I'm Noel Rappin. If you like the podcast and would like to encourage us to continue, please follow us on Twitter at @Tech_Done_Right and leave a review on iTunes. iTunes reviews really help people find the podcast and we really appreciate your time.
	Today on Tech Done Right, we're going to be talking about software, open source and Rails with Eileen Uchitelle and Andrew Horner. Eileen, would you like to introduce yourself?
	
EILEEN:  Hi, I'm Eileen Uchitelle. I am a senior systems engineer at GitHub on the platform systems team. I work a lot on the internals and the externals which includes the GitHub application and the Rails framework. I'm on the Rails core team and the Rails security team. I care a lot about performance and security and open source.

NOEL:  We also have Andrew Horner. Andrew, can you introduce yourself?

ANDREW:  Sure. I'm Andrew Horner. I am a senior developer at Table XI. I work frontend and backend, basically, whatever needs to get done.

NOEL:  Great. Eileen, we want to have you on and we want to talk about Rails core and the work that you do so first of all, how did you get started as a developer and then how did you wind up as part of the Rails core? I know you've worked on Rails for a few years so how did you get involved and how did you come to be on the Rails core team?

EILEEN:  If I tell the whole story about how I became a programmer, it gets really, really long so I will just fast forward to the part where I decided that speaking at conferences was something that I wanted to, even though I found it super terrifying. I thought, maybe I would die if I spoke at a conference. But I also knew, it's really, really important in my career and making myself more visible in the Ruby and Rails communities and blogging was fine and all but I wanted to do more than just that.

I actually gave a talk on ActiveRecord at Mountain West in 2013 or 2014. I can't remember and that's where I met Aaron Patterson and meeting Aaron Patterson was how I started contributing to Rails because apparently, I had found a bug in ActiveRecord and didn't realized it.

NOEL:  I was there.

EILEEN:  Yes.

NOEL:  Yes. I remember this very clearly. You met Aaron because he asked you a question at the end of your presentation and you redirected it back to him. Is that --?

EILEEN:  Yes, he had asked me why "they" don't fix it and I was not "they" at that time he was and I asked, "Well, why don't they fix it?" So we started pairing after that regularly on problems in Rails. First that bug and then other performance related issues. I worked a lot on improving the performance of integration tests. And then I most recently wrote the Rails system testing framework, the Capybara integration. After that, that was enough things.

I did a lot of things and getting on Rails core requires having a well-rounded experience with the framework. You can't really just be into one part of it because core has to make decisions about all sorts of features and the future of the framework. So getting lots of experience in different areas in Rails was part of that.

NOEL:  I don't know a whole lot about how Rails governs. I don't know as much as I should, I guess about how Rails governs itself internally. By being core, what rights and privileges does that give you?

EILEEN:  I'll go over the different levels so then it makes it more clear where some people might be. The GitHub UI doesn't really make it super clear how the differences between the teams and there are big differences. There's the issues team and the issues team has technically full rights because that's how GitHub works. You can't give someone just rights to add the issues. You have to give them full. Or they can only merge documentation pull requests... There's no way for GitHub to know this is a documentation pull request.

NOEL:  I was so tempted to ask you why "they" don't fix it right now.

EILEEN:  Yeah, I know. I'm not on that team. It would be really difficult for GitHub to figure out if this is actually documentation pull request, which issues team is only allowed to merge documentation pull requests and/or triage issues.

NOEL:  Yeah, I'm just messing around. I'm not --

EILEEN:  Yeah, I know. Well, I figured I should explain it before someone "well, actually"s me on the internet. Then the next level is committers and committers have full push access but they don't have rights to release the gems. Some people are going to be confused by that because I did do the actual Rails 5 release last year when I wasn't with the core yet, but that was because I was the release manager.

I had temporary rights to release, so for each release, I would get rights for about 10 minutes and then they would go away. It wasn't like actual rights to release and it had to be given to me by someone each time and then taken away. Now, when you are on core, you have release rights on Ruby Gems without script needing to be run every time. You have access to the all-ever Twitter password which you don't have otherwise. I can just tweet whatever I want now from the Rails Twitter account.

NOEL:  That seems like the kind of power you can use exactly once.

EILEEN:  Yeah and then you also have merge rights on all the repositories that you don't necessarily have just because you have merge rights on Rails. A lot of times you'll get it to the main Rails core and Sprockets when you're a committer. But then when you're a core, then you'll get all the other extra stuff that I don't remember what's there, like the blog and the website and other stuff.

ANDREW:  What's the motivation for the release manager as a separate role? It seems like it would be just as much work to split out privileges for somebody as it is to just release the gems yourselves?

EILEEN:  The role of the release manager is more than just actually releasing the gems. A lot of it has to do with wrangling the people who said they were going to write features or fix bugs before the release because once you release a Beta 1 that's a feature freeze and no new features can go in but you can't beta freeze before there's actually features. Since a lot of people on core and committers and anyone who's writing big features for the next versions of Rails, don't necessarily have full time or any time to work on open source as part of their job of every day so a lot of that is just wrangling and figuring out, "Do we need help on this issue? Can we actually get this feature in?"

A lot of triaging issues and identifying which ones are bugs that are actually related to the Beta 1 release and need to go into the Beta 2 or or RC? What are release blockers? Who's going to write the release notes? Who's going to edit the release notes? Then the final part of it is not just releasing the gems. Actually the easiest part is hitting the release button. It's running the gems locally to test them, building an app locally to make sure that everything just by default works.

For example, right before the 5.0 final release, I found out that SASS Rails or something that isn't one of the gems or maybe it was sprockets or something, needed a new release and didn't work with Rails 5 as it had stood currently so I was at last minute, "Release that gems that way you get it in and then release Rails 5."

NOEL:  One question I have is what is the internal process of deciding what features or what are the priorities for the next version of Rails are?

EILEEN:  We don't meet or have hangouts ever but we do have a chat that's just for people who are on core team. A lot of the feature discussion actually happens in the contributors chat so that everyone who has a commit bit can be involved in that too, including issues team. But the core team is for two things: making those big decisions and in the event that somebody needed to step down or something happened to them, that someone else knew what the future of Rails should be and they were trusted to do that rather than... because David really drives Rails and Rails features.

I think like from the outside, sometimes it feels like, yes, Rails is everybody's framework but Rails isn't necessarily driven by everybody. Everybody can contribute and put their ideas in and contribute back to the framework and try to influence it but really, it comes down to what is David's view of the feature of the framework because it is his framework.

NOEL:  Things get changed in Rails through a couple different routes like people can file bugs, people can file security issues, people can make feature requests. There can be large feature requests that come from David or from Basecamp or are driven by some desire to change the feature of the framework. Let's say, I submit a patch that is not a huge issue. It's just a small feature that I found useful on a number of different applications. What's the process by which I could make that part of Rails -- coming from I'm not a committer -- so what would I need to do and then what would you guys do in response to that to turn something -- let's assume that it's useful -- from an external request or some code that we're running into something that becomes part of the Rails framework.

EILEEN:  First, if you find a security issue, do not open a pull request or issue on GitHub. Please. Nobody. Ever. It's bad because it's public. I'd have to say that because I, also being on a security team that makes me really nervous when someone opens a security issue. If you find a security issue, email Security@RubyOnRails.org or make a HackerOne account and submit it there and we're going to make you go to HackerOne anyway because that's how you get paid. If it's a real security vulnerability, you're going to get paid.

NOEL:  I didn't know any of this. Could you describe what HackerOne is? If I don't know what it is, somebody else out there doesn't know what it is.

EILEEN:  HackerOne is like GitHub for security issues. Everything is private until it's made public. You can report a security vulnerability. Everyone on the security team is notified. We, on the security team, can have a private internal conversation about the issue and how to fix it. We can also communicate with the researcher directly about how to fix it and then maybe send patches back and forth to verify that it's fixed. Then once it is fixed and you go through to the fixed stage, you get paid for finding the vulnerability. Now, we on Rails don't control how you get paid and when you get paid. That's controlled by some other organization that now I'm blanking on.

NOEL:  Okay, so there's another organization that is sponsoring bounties on Rails security issues.

EILEEN:  Yeah.

NOEL:  All right. This is actually something I had no insight into at all. How often do you get genuine security problems coming in through that channel?

EILEEN:  Not super often. It's funny that a lot of them are actually stuff about the certs on RubyOnRails.org but don't necessarily matter but sometimes we get real vulnerabilities and we either don't always know how to fix them. That's just how security stuff goes. A lot of it is about back and forth with security researchers and people that are knowledgeable about security stuff on Rails. Then we have to move our release forward.

There was one vulnerability not that long ago that somebody wrote a blog post about how long it took us to fix but the part of the problem is we didn't know how to fix it. Also you shouldn't be doing that in your Rails app in the first place. It's like one of those two of one-half, half-dozen of the other.

NOEL:  Right. Then there's the internal Rails process of kicking off a patch and kicking off versions for all the releases that are inside the security window that takes place at the end of that.

EILEEN:  Yeah.

NOEL:  Let's say that I have not found a security vulnerability in Rails, which given my own skill set am unlikely to find, I've got some useful add-on and I feel like it should become part of Rails. What should I do in that case?

EILEEN:  Depending on whether or not it's a bug or a feature. If it's a bug you should make a pull request that has a test and has a good reproduction script so that we understand the actual bug and what needs to be fixed. Then open a pull request. We don't need an issue that matches the pull request. It is good to search and make sure that someone else has not done that work before you do it. If your application is on Rails 4 and you're like "I found a bug in Rails 4", make sure it's not on 5 before you spend the time because we're not going to backport to Rails 4. Even if you fix it for yourself, it's not going to make it into Rails codebase, if it's already fixed on master because we don't backport bugs. We're not backporting past 4.2 right now and once 5.1 comes out, 4.2 is not getting backwards either, unless it's a security vulnerability. Bugs only go back one or two releases, I think. That's important to keep in mind so you don't waste your time.

Number two, we don't take what we call cosmetic pull requests so changing all of the hash rockets, which I think we've already fixed that so that we stop getting those pull requests. Changing all the hash rockets... I don't know, changing all of the spellings of color to the European one, which technically the correct spelling in the Rails code base is the English spelling, just as a style guide not as we have determined that US English is the best English. It's just a style guide situation. We don't take pull requests like that.

We have a feature and you haven't written it yet and you're not sure if it's going to be accepted, if you send a message to the Ruby On Rails mailing list, usually we don't take issue requests on the Rails issues tracker on GitHub, mostly because we just don't want it to get bombarded with all the issues people want. And there's a big difference between I want this issue and I want to work on it and I want this feature and I want to work on it and I want this feature and I want Rails core to build it.

NOEL:  Yeah, presumably, you have a better chance of getting it in there if you are saying, "I want this and I'm willing to do it," then...?

EILEEN:  Yeah. The big problem comes in and this is one is like a problem about how much work you do you want to do comes in and that's why we say, if you haven't already done the work, send a message to the mailing list because that's where it can be discussed better with a broader group of people who are not just core and contributors. It gets emailed out every morning or whatever you set on Google Groups settings and then everyone can chime in there.

If you have already built it and you know you want it, you don't want to ask beforehand what should be done. Maybe it's better to show your proof of concept with what have already written, open a pull request, make sure you have lots of details about why you think it's important, what your use case is and stuff like that. Now, depending on who cares most about that part of Rails is going to determine... if it's a really small feature, we have all of the pull request automatically assigned to somebody who has committers' access or push access. But that person might not know whether or not that's a good feature to add.

For example I don't really have that much mentally, don't know that much about active job so I probably wouldn't be merging new features to active job. I would ask someone else like David who knows what he wants for active job because we don't really use it at GitHub and generally it seemed like it works right now so for my purpose, I don't know what features it needs because I don't use it every day so I might not merge something to that.

Whereas because I just wrote system tests, I care a lot more about what the code looks like and what the future features are right now. Especially, since we're not in a final release so I have personally been taking all of those pull requests and deciding whether or not they get merged because that's a feature I wrote and a feature I care a lot about. If you open a routing feature and Andrew White is going to be looking at that pull request because he knows the most about routing. If you opened an ActiveRecord pull request, it could be me or Aaron or Sean Griffin who cares the most about it. It really depends on who's really into that part of the code base.

Especially with the new parts like Action Cable, David is going to care more about new code because it's still pretty pristine and new. The newer parts of Rails, we're going to be more bullish about stock code style and we're going to care a lot more about it being closer to perfect. In ActiveRecord, it's like you know... I love ActiveRecord but it's old so it's harder to be like, "No, don't write the code that way. Write in this other way that's totally different than the rest of the style in all of ActiveRecord --"

NOEL:  The ship is kind of sailed there, is that what you're saying?

EILEEN:  Yeah. It's just it has a slightly different code style so that's what's going to happen when you asked for a feature pull request. Sometimes, it's going to be really hard and you're going to get a lot of comments. I had over one hundred comments on my system test pull request. I know it sucks and it's a lot of work and it's exhausting but it's really worth it once it gets merged. It's hard, especially on a big code base where there's a lot of drive by.

Some people who are going to be talking and fighting against you in your feature or people who have never committed to Rails before and don't necessarily have the same investment than you as someone who wrote a feature does and you have to treat everybody the same because everyone's opinions are valid, even if you don't agree with them.

ANDREW:  Sure. I wanted to touch on something you mentioned sort of in passing which is a backporting policy. The sum of all of my contributions to Rails is one, backport of a bug fix that I think was in Rails 4 but need to be ported to Rails 3.2 around query merges in ActiveRecord. You've mentioned that Rails goes back two versions with backporting but who takes care of backporting? Who makes sure that's all happening?

EILEEN:  Backporting is a little bit not an exact science, mostly because we have to remember to backport. Maybe, this would be a good feature. Sometimes I think... This is me saying this not having any understanding of actually how this code is written but I think it would be really cool feature in GitHub to be able to be like, "Let me backport this thing that I just merged, to this other branch," and if it was clean, that would be easy. But half the time it's not clean. That's what makes it really hard.

A lot of times it just means checking out master, checking out 5.0 stable, backporting the merge commit and pushing which sometimes we don't always... sometimes I merge, then I forget, "Wait, that was actually a bug fix and I need to backport it." One of the hard things right now is that we actually can't merge features because we're in a beta freeze and we don't have a stable branch for 5.1 yet because we're not in an RC so we actually have all of these issues, these pull requests that we're not merging right now because they're features. But it's a manual process and that's what sucks about it. I have often forgotten to go back two versions. I go back one and then realize that I didn't go back both of them or the code has changed so much between 4.2 and 5.0 or like the bug might still be present but it's just not clear how to fix it. A lot of times, if there's really, really big conflicts, then we will ask the person who opened the original pull request to do the backport and open any pull requests for that because sometimes it's just too much to do work manually but yes, it's not an exact science. If it's a bug --

ANDREW:  -There's not a heuristic that you can just wave your magic wand and get everything backported so you kind of handle it case by case.

EILEEN:  Whoever is merging is supposed to be responsible for backporting and then if you can't, you're supposed to add a needs backport label so then someone can backport it for you. Now, there are debates about like the documentation changed. Is it a bug? I consider a documentation and Xavier consideres documentation a bug. It depends like if the documentation was completely incorrect, then that's a bug. If a documentation is sort of incorrect, is it worth backporting two releases, especially if there are conflicts? I don't know. I think generally, it's like a case by case call. You want to be careful that your bug fix or your change that you're making, doesn't change expected behavior in a negative way so that's why it gets hard.

NOEL:  Yeah, especially if you have a complicated framework. Especially, in the case of something like Rails, you almost certainly have cases where things are arguably bugs but people have come to depend on the buggy behavior and fixing the bug will cause problems for them.

EILEEN:  Yeah, I'm hitting that a fair amount. I'm actually working on the GitHub Rails 3.2 to 4.0 upgrade and I'm really close but some of the stuff in there are just like, "This indefinitely reacting to a bug that exists in 3.2 and it's fixed in 4.0 and now I run and like track down what the GitHub fix was and what's the actual fix is and what's the correct behavior and should I keep GitHub code behave like I did in 3.2.

NOEL:  I'm actually in almost the same situation on a client project right now. We're ending on 3.2, at least for the moment. But having the exact same issues of moving a large codebase up through various archaeological layers of Rails history.

EILEEN:  Yeah, it's fun.

NOEL:  Yeah. Fun is.. Sometimes. 

I do want to talk a little bit and you mentioned that the system testing feature in Rails 5.1 which is one of the two or three headline features of Rails 5.1 and the one that you worked on. I like to talk you a little bit about that. Can you talk a little bit about why this feature is getting added and what it does?

EILEEN:  System testing in Rails 5.1 adds Capybara integration into Rails. One of the things that we found when I was at Basecamp was that writing Capybara system tests in Basecamp 3 was a lot of work. We have to set up the Puma servers and we had to set up the browsers and all of these other different things. It started to feel like you're doing a whole lot of work for something that maybe Rails could take on. I think that's the whole point. Like Rails is all about developer happiness and if writing your Capybara test doesn't make you happy, then Rails is not giving you what it needs to. Also back at the RailsConf 2014, David promised that system testing is going to be added to Rails so --

NOEL:  He actually promised a lot of things that you ended up delivering for them because he promised that the integration tests were going to be faster too.

EILEEN:  Yeah, I did all those things for you. Part of the reason that we couldn't add system testing to Rails was because needed to depend on integration test sand integration tests were slow. Also technically, for everyone who know this, ActionController::TestCase is soft deprecated and eventually, someday it will be gone.

NOEL:  Yeah. To define all the terms here, Rails has a couple of different layers of testing. It had what it traditionally called feature testing or controller testing, which did not exercise the entire stack but used a short cut to call controllers directly. Then it also had integration testing, which did go through the routing table like a normal request but it was slow and not tremendously, not super popular I think in the community until the 2014 keynote and the efforts to make it faster and when it really became a priority. You're adding to the existing integration tests default Capybara integration, is that --?

EILEEN:  Yeah, it's a separate class but it inherits from the integration test. Integration test already has sessions working. It already had all the URL helpers working so instead of re-implementing all of that stuff or even half of that stuff, I figured I'm just going to inherit from ActionDispatch integration test and that works. Mostly they didn't get all that stuff, not all the stuff, so you still can't actually access the request and the response maybe one day as another future feature.

NOEL:  You said that one of the goals of this was to make it easier to write the tests and improve developer happiness on the integration tests so what do you do to make it easier to run these kinds of tests?

EILEEN:  For one, this isn't like we don't like Capybara or something. Capybara is super great. The part that makes it hard for junior and mid-programmers to figure out how do I get system tests running. I don't understand like what's this other library, like this is too hard. I just wanted it to work. I don't want to do Rails, generate app and then system tests just work. The goal was to make it work out of the box, with zero configuration, you don't have to do anything, you don't have to do any work, you don't have to figure out like which driver do I want to use, like we defaulted to Selenium for reasons that I'll get you in second.

System testing is basically is like the full browser integration testing. It opens either a headless driver using poltergeist or webkit. If you want to actually see the browser running, you use Selenium and it will actually click all of the buttons and fill in all the fields as if it was a real user doing it but it's a robot so it's super cool. I picked Selenium as a default driver because it's visual. You can see it.

I feel like for new programmers getting into Rails, system tests would be a little bit mysterious. If this thing just ran and took screenshots in the background and you have no idea what it was doing. It's really, really fun and cool to watch Selenium to run to your stuff the first time. It was part of why I picked Selenium. Poltergeist requires installing stuff in home-brew and all these other things that I couldn't make it work out of a box by default. That's why Selenium was a lot easier for that.

Basically, when you generate a scaffold, it generates a system test with it. Now, system tests don't run with your default tests suite if you run bin Rails test. I made it so they don't... Well, I didn't. Actually, Robin did. I was going to do that but he did it first and it was super awesome that he can figure out how to do that. Thank you, Robin if you're listening for doing that.

They don't run in the main test because they can be kind of slow so you have to run bin Rails test system and that will you system tests separately. At GitHub and when I was at Basecamp, we didn't run system tests as part of the main suite except for when we were in CI so that's why we don't do it in Rails because they can be slow and weird. If you do have failing test, the image output is going to just like start appearing in the middle of your test run and you don't want that anyway so that was why.

NOEL:  It integrates Capybara snapshot or something to automatically screenshot failures?

EILEEN:  Oh, yeah. That's the other thing. In Capybara you would have to implement all of that yourself. Go and take a screenshot and do it when it fails. Rails does that by default so it automatically saves a screenshot in your tmp folder and you can look at why did this failed. "Oh, I can see that the name was supposed to be aria but it was actually [inaudible]."

NOEL:  Do you have any idea what you're going to try and do next once Rails 5.1 comes out? What the next feature that you are excited about adding? Or you're kind of on the lookout?

EILEEN:  For me personally?

NOEL:  Yeah.

EILEEN:  I have stuff that I want to fix but I don't know what the undertaking of this is but I don't like how CSRF Protection in Rails behaves. For one, it's part of the callback chain which means that it's order dependent and there are some few, small confusing situations where you can actually remove CSRF Protection from certain actions.

NOEL:  Just to make sure that... CSRF Protection is?

EILEEN:  Cross-Site Request Forgery Protection.

NOEL:  It's a token that Rails expects from forms that are passed to the server.

EILEEN:  In Rails, when you make a request, it checks that the token in the request matches the token in your session and if they don't, it denies the request from occurring and that is so that Rails can prevent someone else from forging your request and making requests on your behalf. Let say, you didn't have CSRF Protection and you had a form for your username and someone was able to send a forged request to change your username, that's like more they need with that. They need to have some access to your session but it's still possible and super scary.

Now, the CSRF Protection in Rails is fine and it works and that's not what's broken about it. What I don't like about CSRF in Rails is it's allowed when it's working so if a token is incorrect, it throws an exception or nulls the session. When it's not working, it's silent. It's not like, "Your CSRF Protection isn't working. And it's not on! Danger! Danger!" It's just silent. It doesn't say anything. That's why I don't like about it because it's the opposite of how security should be. Security should be yelling at you when you're being insecure and quiet when you're not.

I guess it's not exceptional that someone is necessarily trying to forge a request but it is exceptional that you don't have any protection on, unless you absolutely, explicitly didn't want protection on that end point.

ANDREW:  I have definitely run into situations where I'm doing a POST through JavaScript and I forget to send over the CSRF because I'm doing something janky and I just end up signed out and like, "Wait. What just happened?"

EILEEN:  That's just protecting you. At least that's it working. The scary part that comes in is that because of the filters, because of the actions in before filters in Rails are order-dependent. If you were to say like skip CSRF Protection in one controller and then login on another action and your login is depending on your CSRF token, you won't actually be protected on that new route that you added, that you logged in on.

Say, you have a delete request that comes from an email for unsubscribe like, "I don't want CSRF Protection for the first post but I want it for the second post for when I log in." It might not be on. I have slides about it so maybe I'll just give you the slides because it explains it much better. The application controller filters are always going to come first. If you skip one of them, they're not going to be called later in the chain. You have to call it again in the chain because you've already skipped it and it was skipped too early, depending on what your conditionals are.

NOEL:  So you have the possibility of an edge case where you think that the protection is still on but you've actually bypassed it and you don't really know that?

EILEEN:  Yeah. This wouldn't happen somewhere where you're not calling skip_before_filter. But it can also happen in places like gems where certain gems might have controllers that aren't protected because of the order of the filters. What's really scary about it is it's just too quiet. It's not yelling at you when it's not secure. The problem is I don't know how to fix that. I just know that I want to fix it. It has to be something that's backwards-compatible because so many people rely on CSRF Protection. We can't just break everybody's site.

NOEL:  Right, so you have some way where it knows if it's skipped or something?

EILEEN:  I don't know. I don't want to talk about what I think it should be and then people were like, "That's not what you built." I think it's more like you explicitly whitelist actions somewhere and not just in controllers where it might get lost. But we'll see. That's what I like to work on next. I don't know when or if I'll get to work on it but it's something that I've been thinking about for a long time. I know that there are some security researchers that don't think our CSRF Protection is good enough, partially because it's confusing for users so that's something that I would like to work on but it's hard. Security is really hard.

ANDREW:  You brought up soemthing that was kind of interesting which was user expectations because obviously, you're on a really high profile project with Rails. How often do you find potential users' expectations of what's being built influences what you end up having to do.

EILEEN:  You mean on Rails or on --?

ANDREW:  At Rails or any project really.

NOEL:  Yeah. GitHub is a pretty high profile projects --

ANDREW:  That's are sort of open source experience, right?

EILEEN:  This is actually interesting that I want to talk about in my talk. I don't think I mentioned this. I'm speaking on RailsConf on system tests so you should come to RailsConf then you can learn all about it. I'm going to demo it. But then I'm also going to talk a lot about what it means to build a feature in open source and how that's so different from building it like a product feature. Your stakeholders are different, instead of just like, "I've got the CEO who cares about this feature or something." You end up with every CEO in the world caring about your feature.

They all have opinions about how it should be built and what the default options should be and what it should look like and how it should behave and how it should benefit them? That's awesome but also extremely stressful and feeling like you have to please everybody, which is something that I struggle with a lot. When everybody asks for all these changes and if I don't at least respond to them, they're going to think I'm rude or something. When really it's just that I have 135 comments on this pull request and I just can't anymore. There are certain opinions that I have to care more about than others.

NOEL:  This is where some projects use a proposal process, Python has the PEP process and Ember has something where they solicit comment. There's a structured way to solicit comments on a new feature in the design phase and sometimes in the implementation phase. Rails doesn't do that in quite a structured way but it sounds like for the testing thing, you had much the same experience just with a little bit less structure.

EILEEN:  Yeah, we have a lot of internal conversation. Once you open that pull request people start tweeting about it so that brought in a lot more attention, I think than a normal feature requests would have gotten. I hope that no one is sitting at home terrified that if they open a feature pull request, they're definitely getting 150 comments and they just can't deal. That's probably not going to happen. This is a really big pull request with really big... Well, it actually ended up being a lot smaller. It started out a lot bigger than it ended up being. But it was a really big feature that had a lot of opinions behind it. Ultimately, I'm happy with where it ended up so that's something to keep in mind.

I have this funny little analogy about it. If you are building features in open source, it feels a little bit like going to a nice restaurant that has an open kitchen. You're kind of like, "I don't want to see that. I don't want to watch someone make my food right now but I can't look away." Then you start to see the cranky side of open source in people who normally behave really well and you're like, "Wow, this person is just starting to feel the heat right now."

NOEL:  I think that there are certain things that people have attachments to. There are certainly people who have opinions about how a testing framework should work and there are certainly people who have opinions about how security frameworks should work. If you happen to be submitting something that is in the wheelhouse of people that care about it, then you're going to hear from it.

EILEEN:  Yeah. And a big, new testing framework or a big security feature is and should get a lot of comments. I really want the polymorphic types when I ask, "Is this a repository to work by default," which it might now but I'm not working on Rails 4 stuff so I have no idea anymore.

NOEL:  If somebody wants to start contributing to Rails or to an open source project and they maybe don't want to start off in the 150 comment lane, what's the best way for somebody to start to get involved if they feel like they're interested in it.

EILEEN:  Documentation is the most important thing. I don't mean like going through and find that one spelling mistake. I mean looking at how features work and whether or not the documentation for that makes sense especially if you're new to Rails because the documentation is almost always written by someone who wrote the feature and they know it really well so they forget all of the details about like why does Selenium do this thing or whatever. I don't know.

NOEL:  So documentation is a great way. Actually, in some cases being a little bit newer to the framework is an advantage when you're trying to submit these documentation requests.

EILEEN:  Yeah and the other thing is just use the betas. Nobody tests the betas. It's actually really surprising to me because this beta might be one of the most tested betas that we've ever had. All of the new features had a ton of pull requests, fixing things that were broken about them within the first week, which never happens. No one ever tests betas so it's amazing that this beta got super tested so everybody should test the beta releases.

Have a little app that just built, not your production app, I'm not saying that you should... master is super stable. I know that there are apps running on master right now in production... Or maybe not master. Maybe Rails 5.0.1 or whatever but that's still pretty close to master. You don't have to do that but having a test app on the side where you can just upgrade the Rails version and see like, do joins work? Does running the tests work? Does these little things, like my other app that are weird, do they work? Making sure those things just work.

NOEL:  My experience is that by the time Rails gets to beta, Rails itself is pretty solid, not perfect but generally pretty solid. Where you get in trouble if you're trying to build something on the beta is in the larger ecosystem. And the older the gems that you're depend on and the more that you depend on things that don't necessarily interact with Rails that are a little bit more on the periphery, the more likely you are to problems running on early beta.

EILEEN:  Yeah so that's why having a test app makes it a lot easier because you don't have to then worry about that stuff. You can just upgrade all of your gems and call it a day.

ANDREW:  As somebody who is doing a lot of client work and sort of dabbling in open source more than anything. For me my contributions is always come from a place like this thing that I wanted to do that just isn't working so let me figure out how I can fix it in the underlying gem. There was an upgrade for a client recently moving from Rails 3 to Rails 4 and we found the a few incompatibilities and wound up having to strip the entire library and put something else in.

EILEEN:  That's going to be the easiest. A lot of you are asking like, "Where do I start with an issue tracker?" I was like, "If you're starting with someone else's issue that you don't understand, that's going to be frustrating especially if you don't necessarily know how to debug Rails yet. I think it's a lot easier if you have an issue that you found." And I'm not saying that, you can't go with issue tracker. It's just that sometimes it's harder because the people who are opening issues, especially issues that haven't been commented on by anyone from Rails team yet, those are going to be the ones that are missing the most details because that means that no one else has tried running the script yet and maybe, it doesn't even break.

A lot of people don't even write a script so if someone will try to go to fix it before the Rails team finds out, it's not actually a bug. That's why first, don't try to fix issues that don't have a reproduction script because they might not be real issues. If you open an issue, send a reproduction script so that we can test it.

I actually gave a talk in 2015 RailsConf about contributing to Rails. I think most of it is still valid. A lot of it is actually just like a secret debugging talk that was disguised as a contributing to Rails talk. That has a lot about how I debug stuff in Rails which is not necessarily how you would traditionally debug stuff in a regular app. Knowing things like ctags and being able to hook your app up to a local checkout of Rails and then doing a bisect to figure out where the behavior changed or doing stuff like source location. Source location is my favorite tool. You basically can just ask any method in Rails like where it's defined.

NOEL:  Right and pry does that too, if you're using pry.

EILEEN:  Yeah. I'm like a lazy debugger. I always joke that I learned all of my debugging tools from Aaron Patterson but really, I used to do this anyway so it just reinforced the habits that I already had, which just "puts: debugging everywhere.

NOEL:  Yeah, Aaron has a blog post about that too, about just being a puts debugger somewhere. We'll get all of that stuff in the show notes: Eileen's talk and Aaron blog post. Eileen, thank you so much for being with us.

Tech Done Right is a production of Table XI and is hosted by me, Noel Rappin. You can reach me on Twitter at @NoelRap. The podcast is edited by Mandy Moore. You can reach her on Twitter at @DevReps. Our guests today were Eileen Uchitelle who is @EileenCodes on Twitter and Andrew Horner, who I don't think is actually on Twitter in any useful way. You can reach him by screaming really loud, I suppose.

Tech Done Right can be found at TechDoneRight.io or download it via iTunes or wherever you get your podcasts and you can send us feedback or ideas on Twitter at @Tech_Done_Right. 

Table XI is a UX design and software development company in Chicago with a 15-year history of building websites, mobile applications and custom digital experiences. Find us at TableXI where you can learn more about working with us or working for us. And we'll be back in a couple of weeks with the next episode of the Tech Done Right Podcast. Thank you.