Posts in Category: Ruby on Rails

Rspec with Rails 7 and System Tests

Hello world! It’s time for my first post in over 4 years.

I recently set up a new Rails 7 project with Rspec and looked online for tips, as one does. I’ve set up many Rails projects before, but not yet with Rails 7, and it’s been a while. The top result in Google for “rails 7 with rspec” is currently Adrian Valenzuela’s Setup RSpec on a fresh Rails 7 project. His post was really helpful for me shaking off the rust. So rather than writing another post that’s 80% the same, I’ll just share a few additional tips. Think of this post as a companion piece to Valenzuela’s.

Running rails.new

Since you’re using Rspec you can add --skip-test since we don’t need the default MiniTest setup:

rails new your-project -d postgresql --skip-test

Note there is also an option for --skip-system-test but after doing some side-by-side comparisons, I found that if you run --skip-test you are also effectively running --skip-system-test. So if you want system test support – and you probably do – there’s some extra setup you have to do yourself. More on that below.

FactoryBot

Valenzuela notes in a comment on his post that his instructions for adding a spec/factories.rb file are incorrect. But the instructions are still there in the post, so look out for that. You will instead want one file per model in the spec/factories directory (and when you use a scaffold generator it will put factory files there).

System Specs

The configuration shown in his post for spec/support/chrome.rb results in running a headless browser for every system spec, which may be fine for a Javascript heavy site, but could be inefficient for other sites. As explained by Harrison Broadbent in Refactoring from feature specs to system specs:

If we didn’t add this config [see below], RSpec would use selenium for everything. There’s nothing wrong with this… it would just be an unnecessary performance slowdown… rack_test runs a lot faster than selenium, but it doesn’t support javascript. By using rack_test as the default driver for our system specs, they run much quicker. Then we tell RSpec to use selenium for tests that require javascript, since selenium emulates a full browser, and we get the best of both worlds — performance by default, and javascript testing when we need it.

So I’ll recommend using the configuration shown in Broadbent’s post:

#spec/rails_helper.rb

RSpec.configure do |config|
  ...
  config.before(:each, type: :system) do
    driven_by :rack_test # rack_test by default, for performance
  end

  config.before(:each, type: :system, js: true) do
    driven_by :selenium_chrome_headless # selenium when we need javascript
  end
end

If you ran rails.new with --skip-test as I recommended above, you will also need to add the gems shown in Broadbent’s post, since --skip-test also skips the system tests setup (which consists solely of adding these gems):

#Gemfile

group :development, :test do
  ...
  gem "capybara"
  gem "selenium-webdriver"
end

Lastly, I recommend Noel Rappin’s post A Quick Guide to Rails System Tests in RSpec as he has a comprehensive overview and also has tips for updates related to Devise and CircleCI.

RubyConf 2018 is about to start, so let’s talk about RubyConf 2017!

RubyConf 2018 starts tomorrow, and just like I did with RailsConf, I’m very belatedly going to share some highlights from RubyConf 2017, which was in New Orleans last November. It was my first time attending RubyConf, and what struck me the most was the really strong sense of community. Here’s what one first-time attendee had to say:

…This conference was so incredibly worth it. I learned about sweet gems, cool projects, and job opportunities. But more importantly, I met SO MANY totally epic and amazing individuals that even after only three short days I happily now consider friends. I cannot wait to follow their coding lives and journeys in the years to come. I am confident that so many of them are going to do great and groundbreaking things. Plus, I cannot WAIT for my next RubyConf.

That’s from the post 31 thoughts I had while attending my first #RubyConf as an Opportunity Scholar. RubyConf’s Opportunity Scholar program provides financial support for folks who wouldn’t be able to attend otherwise, and are getting started with Ruby. The Scholars are then each matched with a Guide – experienced people who can help them navigate the conference, and make connections for professional development and job opportunities. I applied to be a Guide for this year’s RubyConf and I was selected – I’m looking forward to it!

RubyConf has three tracks of talks, so it’s not possible to attend them all, but here are the ones that were my favorites, including links to the videos for each of them:

  • Live Coding Music with Sonic Pi – this was a really fun talk on Sonic Pi, which Sam Aaron live-programmed while DJing the after-party that night. Here’s video of the talk and a short clip of him DJing:
  • There’s Nothing.new under the sun – this talk includes highlights from some of the best conference talks in the history of Ruby, which required a huge research effort by the presenters. It’s also a great introduction to what makes the Ruby community special. The presenters’ resource list includes links to the talks that the highlighted. Video
  • Code Reviews: Honesty, Kindness, Inspiration: Pick Three – this was my favorite talk, as doing code reviews effectively is one of the greatest challenges teams face, and this talk included a number of innovative and fantastic ideas for doing them well. Video
  • You Are Insufficiently Persuasive – Sandi Metz’ keynote – need I say more? It’s an excellent talk on working well with others: why it’s important, how to do it, and how not to do it. Video
  • High Cost Tests and High Value Tests – an excellent overview of the costs and benefits of different types of tests, and slow tests. Slides | Video
  • Deterministic Solutions to Intermittent Failures – almost all large tests suites I’ve seen over the years have at least some challenges with intermittent failures (flaky tests). This talk consists of hard-won – and refreshingly specific – advice on how to address these challenges. Video
  • Git Driven refactoring – this talk showed me ways of using Git that I’d never thought of before, to make your code better, and also a good introduction to the SOLID principles. Slides | Video

And since the conference was in New Orleans, I now have to show you pictures from some of my time spent outside the conference…

RailsConf 2017 in tweets, and my “Why Do Planes Crash?” lightning talk

RailsConf 2018 starts in exactly one month, and I’m looking forward to it! This means I should probably get around to saying something about RailsConf 2017. The video above is cued to start at the beginning of a lightning talk I gave. The title was “Why Do Planes Crash? Lessons for Junior and Senior Developers.” Analyses of plane crashes show planes actually crash more often when the senior pilot is in the flying seat, often because junior pilots are reticent to speak up when they see problems, while senior pilots don’t hesitate to do so when the junior pilot is flying. There are some great lessons developers can apply from this for how to do mentoring and pair programming.

The lightning talks were at the end of the 2nd day, and I made a last minute decision that morning to sign up and put a talk together. I’ve given a number of conference talks before, but never to a crowd this big, and never with so little time to prepare. Then when it was time to give the talk, there was a technical issue that prevented me from seeing my notes, so I had to wing it. Under the circumstances I think it still turned out ok. Here are my slides (they’re also embedded below) and some tweets about the talk:

I work for ActBlue and we provided Opportunity Scholarships for people who normally wouldn’t be able to attend, for financial or other reasons.

4 of us from ActBlue attended, and my co-worker Braulio gave an impressive full-length talk explaining how our technical infrastructure supports close to 8,000 active organizations, and handles peak traffic like the 2016 New Hampshire primary night, when our traffic peaked at 300,000 requests per minute and 42 credit card transactions per second.

Here are some other highlights from the conference…

Video of Marco Roger’s talk mentioned above.

A group of us took in a Diamondback’s game the night the conference ended, and then the next morning a couple of us headed to the Desert Botanical Garden before flying home.

Lastly, here are the slides from my lightning talk:

Rails and WordPress, BostonRB and WordCamp Boston

I recently moved from Philadelphia to Boston, and my house is currently overflowing with half-unpacked boxes. Despite all the craziness of moving (or perhaps because of it…), I was a speaker at WordCamp Boston this past weekend, and also gave a lightning talk at the BostonRB Ruby meetup last week.

If you’ve followed our blog so far, you may have noticed we talk about both WordPress and Ruby on Rails. While it’s unusual to see a consultancy that develops in these two very different platforms, supporting both gives us the flexibility to choose the platform that best suits our clients’ needs. For applications that primarily need CMS (content management system) functionality, WordPress is a natural fit, while Rails is best suited for highly customized application development. Well known sites with a focus on content, such as The New York Times, CNN, Mashable and many others use WordPress. Twitter was originally developed on Rails, and sites such as Groupon, Bloomberg, Airbnb, and many others also use Rails.

Many consultancies will shoehorn the development of your web application into the one platform they happen to know, even if it’s not a good fit for your needs (this may not be a conscious decision on their part – if they only know one platform well, they may not have the perspective to know whether another platform might be a better choice). For example, WordPress’ Custom Post Types are great for situations where your data can be well represented in the relational database table structure on which WordPress is built, and using them can speed along the development process. But if they aren’t a good fit, then you will likely encounter poor performance when your traffic increases, or have to do custom database development work, which is a breeze in Rails but is awkward and inefficient to do in WordPress.

We also do extensive work in javascript. The ROI calculators we’ve created for Hobson & Co are written entirely in object-oriented javascript, using jQuery and HighCharts (javascript frameworks such as AngularJS or ember.js would have been overkill for this kind of project). Our latest calculator for Greenway Health is a good example.

Regardless of the platform, we take an Agile approach to our work. On the technical side, this means a relentless focus on quality, using object oriented design and test driven development (TDD). My lightning talk at the BostonRB meetup focused on an aspect of this: following the Law of Demeter in Rails application development. Check out my slides.

My WordCamp Boston talk was about the business side of the Agile development process, with a focus on how to build professional, long term partnerships with your clients. I’ve given this talk a few times now, and it’s been a lot of fun to have the opportunity to refine it and keep improving it (I also gave it at the Philadelphia Emerging Technologies for the Enterprise conference and WordCamp Nashville). The video is above, and you can check out my slides.

Here are some tweets from people at each of my talks:

Taming Cybersource: the Cybersourcery Testing gem for Rails

Cybersource is a subsidiary of Visa, and is one of the largest providers of online credit card payment services. As any developer who has worked with Cybersource’s Silent Order POST service can tell you, it’s not the easiest service to work with. It provides a great deal of flexibility, but that comes at the cost of you having to write a good deal of your own code in order to use it. Setting up automated testing is also extremely difficult.

Last year I completed a Cybersource project for the University of Pennsylvania, and that project provided the inspiration for 2 Ruby gems, to simplify working with Cybersource: Cybersourcery, and Cybersourcery Testing. There’s also a demo project, so you can see an example of how to use them in a Rails project.

The readme files provide detailed documentation of their features and how to use them. So rather than repeat that information here, let’s take a look at why these gems are necessary in the first place. There’s a lot to cover, so I’ll discuss the testing gem in this post, and Cybersourcery in the next one.

Writing tests that can be repeated and automated provides benefits such as improving the design of your code (if you’re doing test-driven development) and catching regressions early (when changes to your code inadvertently introduce bugs). This can be challenging with 3rd party services, as we don’t want to call those services every time we run our test suite. VCR is a gem that helps with this problem: by recording requests and responses, it allows you to play back those responses in your tests, rather than making real-life calls in every test run.

Unfortunately, Cybersource makes this kind of testing especially difficult. There are 3 different servers involved in processing a transaction through Cybersource, and the key difficulty is that one of them is at a fixed URL that is not easy to swap out in the test environment. Cybersource calls this URL the “Customer Response Page.” It is saved on the Cybersource server as part of the merchant profile, so it cannot be updated dynamically. If you are a developer attempting to test Cybersource transactions, this diagram illustrates the scenario:

                     +                    +                   +
                     |     Developer's    |    Cybersource    |  "Customer Response"
     User's browser  |     test server    |    test server    |         server
  +------------------+--------------------+-------------------+---------------------+

    Request credit
      card form
          +
          |
          +----------->   Respond with
                        credit card form
                               +
                               |
     Submit form <-------------+
          +
          |
          +------------------------------>   Process and log
                                              transaction;
                                            generate "Customer
                                              Response" form
                                                    +
                                                    |
    Hidden "Customer <------------------------------+
    Response" form is
      automatically
     submitted by JS
          +
          |
          +--------------------------------------------------->   Process submission;
                                                                 generate response page
                                                                         +
                                                                         |
        Display  <-------------------------------------------------------+
     response page

So, what the heck is going on here? The first few steps makes sense, but then when you submit the credit card payment form to Cybersource, things start to seem strange. What happens is that Cybersource sends a seemingly blank page to your browser. But it only appears for a second, as it contains a hidden form, which is immediately and automatically submitted to the “Customer Response Page.” This is the page where users are sent when transactions are complete. You provide the URL for this page when setting up your merchant profile in the Cybersource Business Center. This is a page you create and host yourself – you can use it to show users a “thank you” message, log information about the transaction, etc.

So why doesn’t Cybersource simply redirect to your response page after processing the transaction? Why this peculiar reliance on a hidden form? The reason is that conventional redirects use the GET method, which is meant for idempotent requests. An idempotent request is one that can be safely repeated, which definitely does not apply to a credit card transaction, or logging it. So Cybersource’s forms appropriately use the POST method, which is meant for non-idempotent requests. This is why, if you submit a POST form, and then click “back” in your browser, and try to submit the form again, your browser will warn you, and ask if you really want to submit the form again.

In the case of Cybersource, this is a thorny problem. Trying to do a POST redirect has issues, for these reasons. A redirect isn’t really appropriate anyway: the Cybersource server does some work when it receives the user’s credit card submission (charging the user’s card), and then your response page may also do some work when it receives the hidden form submission (such as logging details of the transaction). These are distinct activities, so – while having two forms may seem odd – it’s a viable solution. Cybersource came up with this before asynchronous requests were a common practice (which is a big part of the reason it’s harder to work with than newer services like Stripe).

The Cybersourcery Testing gem makes it possible to set up automated, repeatable tests in this complex environment. It provides a “translating proxy” server, running on Sinatra, which has middleware to manage the requests and responses between the 3 servers involved in Cybersource transactions. Middleware is especially useful in this situation, as it allows us to modify requests and responses before they reach the application’s code.

In order to explain how the gem works, let’s first take a look at its dependencies:

  • The Rack Proxy gem is a Rack app which allows you to rewrite requests and responses. It’s very open-ended and is designed to be subclassed, so you can implement rewrite rules appropriate to your project.
  • The Rack::Translating Proxy gem inherits from the Rack Proxy gem, and provides an implementation that’s suitable for the Cybersource use case. We need to provide it with a target_host URL, which indicates where the proxy server should redirect requests. We also need to provide it with a request_mapping, which indicates what strings to find in the requests and responses, and what to change them to. It uses a hash format, so that on requests, the keys are translated to the values, and on responses, the values are translated to the keys.

The Cybersourcery Testing gem inherits from the Rack::Translating Proxy gem, and implements the methods described above. Specifically:

For the target_host, we provide the URL of the Cybersource testing site. So if the proxy server is running at http://localhost:5556, and the target_host is https://testsecureacceptance.cybersource.com, requests to http://localhost:5556/some/path will be redirected to https://testsecureacceptance.cybersource.com/some/path. The gem also hooks into VCR, allowing us to record transactions that pass through the proxy server, for use in subsequent automated tests.

This is a simplified version of the request_mapping implementation, using hard-coded values for clarity:

def request_mapping
  {
    # local test server                Cybersource's "Customer Response Page" URL
    'http://localhost:1234/confirm' => 'http://your-site.com/confirm'
  }
end

A Cybersource transaction in this environment looks like this:

  1. The credit card form is submitted and the proxy server receives it.
  2. Based on the target_host, the proxy server passes the request through to the actual Cybersource test server. If the transaction was previously recorded with VCR, VCR will instead play back the recording of the transaction.
  3. Cybersource (or VCR) sends a hidden form back to the browser, which is automatically submitted via JavaScript to the “Customer Response Page” URL. The middleware’s request_mapping will rewrite the URL of the form’s action, causing the form to instead submit to the local test server.

The upshot is, the gem handles all this complexity so you don’t have to. By following the setup steps in the readme, you can get a robust test environment setup for Cybersource without breaking a sweat. The Cybersourcery Testing gem offers other features as well, such as reporting of undocumented Cybersource errors. Check out the README to learn more!

Get some more coal, it’s time to fire up the old blog engine

I haven’t been blogging for ages – it’s time to fix that. Let’s start with a recap of last night.

The monthly philly.rb meetup at the Comcast Center: how good were the presentations? They were just as good as the view from our room on the 45th floor:

View from the 45th floor of the Comcast Center, where we had the philly.rb monthly meetupView from the 45th floor of the Comcast Center, where we had the philly.rb monthly meetup
View from the 45th floor of the Comcast Center, where we had the philly.rb monthly meetup11-Feb-2014 17:58, HTC EVO, 2.0, 3.63mm, 0.067 sec, ISO 1209
 

I got a lot out of Nate Olds’ talk, “Refactoring with a View.” A large part of my career has been spent wrestling with big, old, sprawling, messy codebases, so Nate’s real-life walk-through of his strategies for dealing with such challenges was very informative. Check out the February meetup page for more information.

Afterwards, a group of us went to Ladder 15 for beer, where we met up with Maria. She stoically suffered through several minutes of extremely nerdy conversion, before she and I headed to the Boot & Saddle, for a rare evening out without the boys, to see one of Maria’s favorite bands, Cibo Matto. It was a sold out show, and we ended up stuck near the back, but it was still a good time. I tried to take a couple videos, but they didn’t come out. A good introduction to their utterly goofy side is the song Sci Fi Wasabi (I’m Miho Hatori, straight out of purgatory), and their split-screen video for Sugar Water, with one side portraying the visual story in reverse, is quite clever (the ending is in the middle).

I’m presenting at the WordPress ‘Burbs meetup next Monday on Kanban. And on March 1 I’m presenting at WordCamp Lancaster: A11Y? I18N? L10N? UTF8? WTF? Understanding the connections between accessibility, internationalization, localization, and character sets.

Talking to strangers causes train wrecks

I started attending the philly.rb meetups in the early Fall, and I did my first lightning talk last night (before the scheduled main speaker, anyone can come up and give a talk that’s 10 minutes or less). There’s a big debate right now in the Rails community on the “right” way to design applications. As a relative newcomer to Rails, I’ve been trying to find the right balance between leveraging my previous experience and habits from working in other languages and frameworks, vs. the opportunities Rails provides to approach application design in a different way (the “Rails way”).

My talk takes a look at one aspect of that debate – how to apply the Law of Demeter to Rails development. I’ve given many long talks before – this was my first short one. It was actually harder, especially since I delved into theory, but it went well. I was going to write a summary this morning, but @trevmex already did.

Here are my slides (direct link):

Talking to strangers causes train wrecks from mtoppa