Oh no my "secret_token" is exposed on the Internet!

What does this mean?

The "secret_token" is used as a key to cryptographically ensure that no one tampers with your Rails apps' session cookies. So if this token is exposed on the public Internet, attackers are able to forge session cookies which are valid within your Rails application. This does not only mean that likely an attacker can impersonate any user of your application. If the bad guy is a bit more fancy she (or he) can craft some session cookies which will make the Rails app execute arbitrary code. This attack applies if you use the standard Ruby on Rails session cookies. If you are using ActiveRecord sessions, you are most likely fine =).

What should I do?

You should replace the "secret_token", obviously. In order to keep the new token secret this time, you could do the following (code stolen from Gitlabhq's config/initializers/secret_token.rb):
# Be sure to restart your server when you modify this file.

require 'securerandom'

# Your secret key for verifying the integrity of signed cookies.
# If you change this key, all old signed cookies will become invalid!
# Make sure the secret is at least 30 characters and all random,
# no regular words or you'll be exposed to dictionary attacks.

def find_secure_token
  token_file = Rails.root.join('.secret')
  if File.exist? token_file
    # Use the existing token.
    # Generate a new token of 64 random hexadecimal characters and store it in token_file.
    token = SecureRandom.hex(64)
    File.write(token_file, token)

YOUR_RAILS_APP::Application.config.secret_token = find_secure_token
Where YOUR_RAILS_APP should be adjusted accordingly. Second step would be to exclude the file ".secret" form your Git repository by listing it in your ".gitignore" file. If your Rails app has a larger user base, you should think about issuing a security advisory in order to alert your users about this issue.