New line characters in textareas

I’m working with a textarea where I take its input and pass it to an external service, which will validate the content.

In tests, I used webmock to stub out this external service. It worked fine with integration tests, but using Rails system tests, the stubs didn’t work.

Turns out, the default new line character in textareas, according to the HTML standard, is \r\n. In the stubs, I used \n.

This was somewhat unexpected. Windows also uses \r\n as its new line character, but the tests were not passing on Mac OS X, so I didn’t think it was something related to the textarea.

2022-06-15 by Andy Pfister

Ecto stores timestamps as :naive_datetime

When you use your timestamps from your Ecto repository to show your user edit or store times of database elements, it might make sense to convert them to a specific time zone. Otherwise the user may feel that time has been saved incorrectly.

By default, Ecto creates and stores timestamps in :naive_datetime format, for reasons of backward compatibility.

:naive_datetime is like :utc_datetime, but without timezone information.

So, how to display a :naive_datetime in a time that is relevant to the user? Within one of our larger repositories using Elixir on the backend and React on the frontend we use the daysJS library to assist with date transformation in Javascript.

Import these libraries:

import dayjs from "dayjs";
import utc from "dayjs/plugin/utc";
import tz from "dayjs/plugin/timezone”;

Set up daysJS:

dayjs.extend(utc);
dayjs.extend(tz);

// you can define a specific time zone, or guess the user's time zone:

dayjs.tz.guess();

Then, when we pass the :naive_datetime into a dayjs wrapper, we need to append a “Z”, so that it knows to be time zone aware when displaying this time, for example as in here:

{dayjs(`${YourNaiveDatetime}Z`).format("DD.MM.YYYY H:mm")}

And voila. Your user now see this time stamp in a time meaningful to them, in relation to their local timezone.

2022-05-26 by Andrea Kreideweiss

Running Molecule scenarios in parallel

If you test Ansible roles with Molecule, you sometimes have multiple scenarios (e.g. for different application versions that share the same role).

Previously, it wasn’t possible to run all scenarios at the same time as the created Docker containers were always named the same, despite different scenario names.

If you execute Molecule with --parallel, it’ll create UUIDs for the container name, which enables parallel execution.

Example:

molecule test --scenario-name 5.5 --parallel
2022-04-14 by Andy Pfister

Rails DateHelper translations

An interesting bug occured to one of our projects: today, on the 1st of March, some system tests started failing. The reason: difference in translation of the timestamp. What has been translated as:

  The questionnaire was not completed yet. Proceed ... (2 months)

started to be translated as

  The questionnaire was not completed yet. Proceed ... (about 2 months)

Tested objects were created with a timestamp 2.months.ago => today it means Sat, 01 Jan 2022. In a view this timestamp is translated with the date helper time_ago_in_words. In the documentation I found that a threshold between about 2 months and 2 month is 59 days.

Since February has 28 days and January 31 days , the timestamp in tests is equal to 59 days. It seems like documentation is not precise enough though:

From docs:

  44 days, 23 hrs, 59 mins, 30 secs <-> 59 days, 23 hrs, 59 mins, 29 secs  
 # => about 2 months
  59 days, 23 hrs, 59 mins, 30 secs <-> 1 yr minus 1 sec
 # => [2..12] months

From console:

>> 2.months.ago
=> Sat, 01 Jan 2022 10:25:48.330338000 CET +01:00
>> 31 + 28
=> 59
>> time_ago_in_words(59.days.ago)
=> "about 2 months"
>> time_ago_in_words(60.days.ago)
=> "2 months"

59 days is still translated as about 2 months and everything above as %{count} months.

The fix is to change the timestamp in tests to not rely on this translation difference.

2022-03-01 by Tatiana Panferova

Ruby argument forwarding

Ruby supports argument forwarding using the ellipsis operator ... called “all arguments splat” (introduced in version 2.7)

# test.rb

def a(...)
  b(...)
end

def b(first:)
  puts first
  yield
end

a(first: "first") do
  puts "from a block"
end
$ ruby test.rb
first
from a block
2021-12-22 by Dimiter Petrov

method: :delete on link_to in Rails 7

From Rails 4 (?) until Rails 6.1, you have Rails UJS in both default setups for Webpacker and Sprockets available. UJS does some magic left and right. One of that magic is the following: You can define a link_to element in your code as follows:

= link_to "Löschen", server_path(server), method: :delete, class: "pl-8"

This will turn into the following HTML:

<a class="pl-8" rel="nofollow" data-method="delete" href="/servers/c6a580a4-f855-5413-9181-70f1a5a256e9">Löschen</a>

Upon clicking this element, Rails UJS will intercept the request and replace the link with a proper form in case data-method is set. You can check out the relevant code here.

If you start a new Rails 7 app with importmaps, UJS won’t be there. You could add UJS via the import maps, but you can actually use button_to to solve this scenario.

= button_to "Löschen", server_path(server), method: :delete, form_class: "pl-8 inline"

This will result in the following HTML:

<form class="pl-8 inline" method="post" action="/servers/c6a580a4-f855-5413-9181-70f1a5a256e9">
  <input type="hidden" name="_method" value="delete">
  <button type="submit">Löschen</button>
  <input type="hidden" name="authenticity_token" value="dcOVJmD5M7FX5Iu1RsLj5exziAJ8Z9aKCxs2AITUXvX-mbemDEzdbMX2-6SOu8_AliDGlSHEmRiT1ahcOkviNQ">
</form>

As you can observe, Rails will insert an entire form when declaring button_to. With form_class: inline we can ensure that the form will be inlined as a normal link would as well.

2021-11-02 by Andy Pfister

Installing Ruby <2.4 on a modern Ubuntu

I sometimes use Ubuntu 20.04 for work. Today I had to install Ruby 2.2.10 for a client project, which wasn’t able to compile from source.

Apparently, you need your system to have OpenSSL 1.0 and GCC 6 present - 20.04 ships per default with GCC 7 and OpenSSL 1.1.

echo "deb http://dk.archive.ubuntu.com/ubuntu/ bionic main universe" | sudo tee /etc/apt/sources.list
sudo apt update && sudo apt install libssl1.0-dev g++-6
CC=$(which gcc-6) rbenv install 2.2.10

Credits:

2021-06-07 by Andy Pfister

All day, week, month, quarter, year... in Rails

Rails has some fancy helpers on date and time objects:

date = Time.zone.today

date.all_day
# => Wed, 12 May 2021 00:00:00.000000000 UTC +00:00..Wed, 12 May 2021 23:59:59.999999999 UTC +00:00

date.all_week
# => Mon, 10 May 2021..Sun, 16 May 2021

date.all_month
# => Sat, 01 May 2021..Mon, 31 May 2021

all_quarter
# => Thu, 01 Apr 2021..Wed, 30 Jun 2021

date.all_year
# => Fri, 01 Jan 2021..Fri, 31 Dec 2021

https://api.rubyonrails.org/v6.1.3/classes/DateAndTime/Calculations.html

2021-05-12 by Mario Schüttel

Rails `link_to` with a new ActiveRecord-object takes you to the index page

link_to with a persisted object takes us to the show page of the corresponding object:

@team = Team.first
link_to 'Cancel', @team
# => <a href="/teams/123">Cancel</a>

Today I learned that if the object is a new_record? the created link takes us to the index page of given Model:

@team = Team.new
link_to 'Cancel', @team
# => <a href="/teams">Cancel</a>

This is super handy for a “Cancel” button in a form, that usually should behave differently if the object is or is not persisted.

2021-04-01 by Mario Schüttel

Format currencies for display with JavaScript

Number.toLocaleString() can format a number according to the browser locale, or a specific locale.

18.95.toLocaleString("en")
// "18.95"

18.95.toLocaleString("fr")
// "18,95"

Integer literals need to be explicitly typed:

1000.toLocaleString()
// throws an error

Number(1000).toLocaleString('ch')
// returns "1’000"

Number(1000).toLocaleString('en')
// returns "1,000"

You don’t need the number constructor when using a variable:

let n = 1000
n.toLocaleString("en")
// "1,000"

It gets interesting when you add more options:

8.95.toLocaleString("en", {
  style: "currency",
  currency: "CHF"
})
// "CHF 8.95"

Number(2000).toLocaleString("en", {
 style: "currency",
 currency: "CHF"
})
// "CHF 2,000.00"

Number(2000).toLocaleString("en", {
 style: "currency",
 currency: "CHF",
 minimumFractionDigits: 0
})
// "CHF 2,000"
2021-03-12 by Dimiter Petrov

'yes' and 'no' keys in Rails translations yaml

yes and no represent true and false in Yaml (see the yaml spec). The Ruby yaml parser has tests to make sure yes and no get evaluated as booleans.

That means you need to put these keys in quotes in translation files:

en:
  'yes': Yes
  'no': No

Without the quotes you get a missing translation error when trying to access them with I18n.t('yes') or I18n.t('no')

2021-03-09 by Dimiter Petrov

Sort git branches by most recently modified

I sometimes forget the name of the git branch I was working on recently.

git branch -a lists all branches in alphabetical order by default. That is impractical if there are many branches.

This is where the --sort flag comes in handy. The following will sort branches by commit date in reverse chronological order.

git branch --sort=-committerdate

committerdate is the sort criterion. The minus indicates descending order.

This by itself can be hard to remember, but you can save the command in a shell alias. You can also set the default branch sort order with the branch.sort key in the git configuration.

2020-11-16 by Dimiter Petrov

Click to open a URL from the macOS Terminal app

Sometimes you read text in the terminal that contains a link. To access it, you could select the text, copy it and paste it in a browser.

Some terminals let you skip these steps and directly open the link when you click on it.

It turns out the default terminal application on macOS, Terminal, has that capability, too, but it’s hidden.

Press Cmd and double click on the URL.

2020-09-30 by Dimiter Petrov

Change locale of Postgresql database

I had an issue with search implementation in my app: search for records in Russian worked, but was case sensitive. I found out, that I have to change locale of my database (specifically it’s categories LC_COLLATE and LC_CTYPE) to ru_RU.UTF-8. But you can’t change it for existing database - locale should be defined when database gets created. So the steps should be:

  1. drop existing database
  2. check list of locales of your system: type locale -a in your terminal, find the name of locale that you want to change to
  3. add to you database.yml following configuration:
default: &default
  adapter: postgresql
  encoding: utf8
  ctype: ru_RU.UTF-8 
  collation: ru_RU.UTF-8
  template: template0

value of ctype and collation should be the same as the locale name in your system.

After that problem was solved locally, but code didn’t pass Semaphore pipeline. After mailing with their support and adjusting proposed solution, I did following:

  1. added Dockerfile to the repo with following congifuration:
FROM postgres:9.6
RUN localedef -i ru_RU -c -f UTF-8 -A /usr/share/locale/locale.alias ru_RU.UTF-8
ENV LANG ru_RU.UTF-8
  1. added a job in semaphore.yml to re-create postgres docker image with new locale:
- docker build --tag postgres:lang .
- docker run --rm --net host -d -e POSTGRES_PASSWORD=semaphore --name postgres -v /var/run/postgresql:/var/run/postgresql postgres:lang

Now it works properly. For more information about locale support check https://www.postgresql.org/docs/12/locale.html

2020-06-03 by Tatiana Panferova

Random with gastsby and react

The Problem:

On our website we have a carousel that should show items of a randomly sorted array.

The problem now was, that text was displayed of the randomly chosen element during runtime. But the attributes of a tag had the value of the randomly chosen element of the build time of gatsby.

That behaviour led to inconsistent ctas. That means that a link of “Bereit für die Informations-Drehscheibe der Zukunft?” pointet to ‘calendly’ instead of ‘/hub’.

The solution:

Add the randomly sorted array to the local state of the class in componentDidMount and read it from there later on.

2020-05-14 by Mischa Steiner

Proxy Exclusion on Linux

What I already knew, was that the environment variable proxy_http and proxy_https lets you set a proxy server on Linux.

This redirects all connections to the specified hosts. But sometimes, you don’t want this, e.g. when accessing internal resources. So you can configure a variable named no_proxy to exclude those:

export no_proxy="internal.stuff"
2020-05-06 by Andy Pfister

Syntax highlighting of code blocks with markdown

When you write comments or Readme on github, you can add an optional language identifier to enable syntax highlighting.

For example, for Ruby code:

```ruby
def initialize(budget:, start_date: Time.zone.today, end_date:)
  @budget = budget
  @end_date = end_date
  @start_date = start_date
end
```

and the result will be:

Imgur

2020-03-10 by Tatiana Panferova

Ignore whitespace when looking at a diff

Some pull requests look terribly messy, especially if most of the changes are due to changing the indentation level.

To look at these diffs in a nicer way, you can use git diff -w, which ignores whitespace.
This allows you to see the more interesting changes much easier.

This also works on GitHub and GitLab.
Just add ?w=1 at the end of the URL.

2020-02-20 by Florian Zinggeler

Filter finding files by creation date

Find files that were created within the past 7 days

$ find /path/to/dir/ -mtime -7
./file_from_yesterday
./file_from_three_days_ago

Find files that were created until 7 days ago

$ find /path/to/dir/ -mtime +7
./file_from_ten_days_ago
./file_from_thirty_days_ago
2020-02-07 by Mario Schüttel

Docker swarm keeps a tasks.db file which can grow large...

Some of our deployments suddenly failed mysteriously: docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:319: getting the final child's pid from pipe caused \"EOF\"": unknown.

Various commands showed something was wrong:

# docker service ls
insdns4o4bu8        xxx_1     replicated          0/1           xxx   *:8000->8000/tcp
dggj6i4actyn        xxx_2     replicated          0/1           xxx   *:8001->8001/tcp
# docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS
ux3enftr6krhlrp8bbodo2q1l *   wsdocker6           Down                Active              

systemctl status docker finally gave the right hint:
swarm component could not be started before timeout was reached

Which sent me to this issue comment which finally resolved the issue.

Turns out docker swarm has a tasks.db file which can get corrupted somehow.

2020-01-29 by Florian Zinggeler

Migrate from MySQL (or MariaDB) to Postgres

There’s a neat tool for macOS (maybe others too) to migrate a MySQL/MariaDB database to Postgres:

brew install pgloader

Here’s how I used it then:

createdb my_database_name
pgloader \
  mysql://root@localhost/my_database_name \
  postgresql://localhost/my_database_name

Very simple. It migrated a MariaDB 10.2 database to Postgres 12 without any issues.

2020-01-23 by Mario Schüttel

Debug Docker build steps

If a Docker build fails, you can inspect the intermediate steps.

The Rails asset compilation step below fails:

Step 10/12 : ADD . $APP_HOME
 ---> af4f399351dc
Step 11/12 : RUN bin/rake assets:precompile
 ---> Running in b5d9eec28028
...

We can run a command in the context of the last successful step:

docker run -it af4f399351dc sh
2020-01-17 by Dimiter Petrov

"not"-scopes in ActiveRecord::Enum

ActiveRecord::Enum got a nice new feature in Rails 6.

class User < ApplicationRecord
  enum status: [:active, :inactive, :radioactive]
end

The Model automatically receives these methods

User.active             # where(status: 0)
User.not_active         # where.not(status: 0)

User.inactive           # where(status: 1)
User.not_inactive       # where.not(status: 1)

User.radioactive        # where(status: 2)
User.not_radioactive    # where.not(status: 2)

The Model’s objects get these methods (unchanged since Rails 5)

user.active?        # status == 0
user.active!        # update!(status: 0)

user.inactive?      # status == 1
user.inactive!      # update!(status: 1)

user.radioactive?   # status == 2
user.radioactive!   # update!(status: 2)

https://api.rubyonrails.org/v6.0.0/classes/ActiveRecord/Enum.html

2019-12-19 by Mario Schüttel

Create a working copy from a bare git repository

Bare git repositories don’t have a working copy where you can read and edit files. Instead it contains what is usually in the .git subdirectory of a repository. You can re-create a “traditional” repository by cloning the bare repository.

git clone --local bare-repo-path project-name
2019-12-10 by Dimiter Petrov

Ruby: Use cover? instead of include? for ranges

include? compares every object of an array, while cover? checks if an item fits between the end points.

This difference makes it much faster to use cover? for ranges.

Example:

    one_hundert_years_ago = Date.today - 100.years
    today = Date.today

    t = Time.now
    (one_hundert_years_ago..today).include?(Date.tomorrow)
    puts Time.now - t
    # => 0.029056

    t = Time.now
    (one_hundert_years_ago..today).cover?(Date.tomorrow)
    puts Time.now - t 
    # => 3.4e-05 (= 0.000034)
2019-11-26 by Mischa Steiner

git stash --include-untracked

Let’s say you have modified a file in a git repository and have also created a new file, but not added it to the repository yet:

$ git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   README.md

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    notes.txt

no changes added to commit (use "git add" and/or "git commit -a")

If you run git stash, the untracked file remains in the working copy:

$ git stash
Saved working directory and index state WIP on master: 0f25a2b Initial commit
$ git status
On branch master
Untracked files:
  (use "git add <file>..." to include in what will be committed)

    notes.txt

nothing added to commit but untracked files present (use "git add" to track)

Chances are the untracked files are related to the change you are trying to make and would like to bundle them in that stash.

The --include-untracked option (or -u in its short version) allows you to do just that:

$ git stash -u
Saved working directory and index state WIP on master: 0f25a2b Initial commit
$ git status
On branch master
nothing to commit, working tree clean

That’s a nice alternative to adding the untracked files and making a “work in progress” commit.

2019-11-26 by Dimiter Petrov

ActionDispatch::SystemTestCase no longer inherits from ActionDispatch::IntegrationTest in Rails 6.0.1

With Rails Release 6.0.1, ActionPack comes with this breaking change:

ActionDispatch::SystemTestCase now inherits from ActiveSupport::TestCase rather than ActionDispatch::IntegrationTest

Link to CHANGELOG

That means that some useful helper methods won’t be available anymore in system tests.

In one project we use a method fixture_file_upload that was affected by this change. We brought this back by including the corresponding module in ApplicationSystemTestCase:

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  include ActionDispatch::TestProcess::FixtureFile

  # ...
end
2019-11-08 by Mario Schüttel

Adding a limit to ActiveRecord "last" method

ActiveRecord supports adding a LIMIT to your SELECT statement through the last method:

User.last(10)

results in the following SQL query:

User Load (0.4ms)  SELECT `users`.* 
FROM `users` ORDER BY `users`.`id` DESC LIMIT 10
2019-10-02 by Antonia Horincar

Explicitly mount files with docker-compose

When mounting files in docker compose, like this:

volumes:
  - ./file-to-mount:/target/file-to-mount

an empty directory is created inside the container, if the file-to-mount does not exist! This leads to annoying debug sessions…

One can use the following syntax (with docker-compose 3.4), to mount a file more explicitly:

    volumes:
      - type: bind
        source: ./file-to-mount
        target: /target/file-to-mount

Now, when the file does not exists, it is not possible to start the container.

invalid mount config for type "bind": bind source path does not exist...
2019-09-18 by Philipp Koster

Run command line scripts from within Rake tasks

My usual approach when writing Rake tasks is to import a library and then use its API:

require_relative '../slides'

namspace :slides do
  desc 'Create an empty slideshow'
  task :new do
    Slides.new.write_to_disk
  end
end

Sometimes that is impractical: the API is not public, or you need to invoke a standalone script which only has a command line interface.

I found out Rake implements a ruby method which lets you invoke the ruby program directly:

task :new do
  ruby 'create_slideshow.rb'
  puts 'Created a new slideshow.'
end

There is also a sh method, which allows you to run system commands. Make sure to use the multiple argument version and avoid passing it unfiltered user input.

2019-09-11 by Dimiter Petrov

Install mysql gem on Mac OS X

According to this Gist.

brew install mysql
brew install openssl
gem install mysql2 -v '0.5.2' -- --with-ldflags=-L/usr/local/opt/openssl/lib --with-cppflags=-I/usr/local/opt/openssl/include

You need mysql and not mysql-client, otherwise the gem compilation process will complain.

2019-09-05 by Andy Pfister

Step through migrations in Rails

tl;dr db:forward is the opposite of db:rollback

bin/rails db:migrate

runs all pending migrations.

If you want to migrate to a specific DB version, you can use

bin/rails db:migrate VERSION=20190730123456

which is rather cumbersome, especially if you only want to run one migration. Therefore you can use

bin/rails db:forward

You can combine it with variable STEP to execute multiple migrations.

bin/rails db:forward STEP=5
2019-08-02 by Mario Schüttel

rubocop-rails

rubocop 0.72.0removes all Rails cops, since they “[…] want RuboCop to be focused just on Ruby […]” (and I agree with them), see PR #5976.

In order to use the Rails cops after rubocop 0.72.0, we need to add rubocop-rails gem and require it as a dependency in .rubocop.yml:

require:
  - rubocop-rails
2019-07-29 by Mario Schüttel

Enumerable#grep

Ruby enumerables respond to grep. You can use it to quickly filter a collection with a regular expression.

['Marta', 'Mario', 'Bill'].grep(/Ma/)
# => ['Marta', 'Mario']
2019-07-12 by Dimiter Petrov

Update nested maps in Elixir

Being a functional programming language, Elixir is a bit different from other languages when it comes to update a nested data structure.

For example, in javascript we could do:

data = {
  inner: {
    one: {
      a: 1
    },
    two: {
      b: 45
    }
  }
}

data.inner.one.a = 2

In Elixir you have to build a new map with the updated information, for example:

data = %{
  inner: %{
    one: %{
      a: 1
    },
    two: %{
      b: 45
    }
  }
}

new_one = %{data.inner.one | a: 2}
new_inner = %{data.inner | one: new_one}
new_data = %{data | inner: new_inner}

which is not very handy.

In other functional languages like Haskell, there are libraries like Lenses that aims to solve the problem. In Elixir the kernel have an put_in function that acts in a similar way:

data = put_in(data, [:inner, :one, :a], 2)

You can find other similar functions in the Kernel documentation

2019-06-13 by Tiziano Puppi

The query function in Ecto Repo (Elixir)

There is a hidden function in Ecto which is the function query in the Repo module.

This is a wrapper around the Ecto.Adapters.SQL.query and it is injected in the Repo module of your application. It can be called in this way:

MyApp.Repo.query("SELECT * FROM mytable")

This function can also take an array of query parameters like:

MyApp.Repo.query("""                                           
SELECT * FROM mytable
WHERE id = $1
""", [42])

This is quite handy when you need to debug in production, and there is no database client available.

2019-06-07 by Tiziano Puppi

Union, Difference and Intersect with Ruby Arrays

Union

> [:a, :b, :c].union([:b, :c, :d])
 => [:a, :b, :c, :d]
# or 
> [:a, :b, :c] | [:b, :c, :d]
 => [:a, :b, :c, :d]

“Union all” / Concat

[:a, :b].concat([:b, :c])
 => [:a, :b, :b, :c]
# or
[:a, :b] + [:b, :c]
 => [:a, :b, :b, :c]

Difference

> [:a, :b, :c].difference([:b, :c, :d])
 => [:a]
# or
[:a, :b, :c] - [:b, :c, :d]
 => [:a]

Intersect

> [:a, :b, :c] & [:b, :c, :d]
 => [:b, :c]
2019-06-07 by Mario Schüttel

The sample output HTML element

There is an HTML element called <samp> meant for representing computer program output. The specification document has this humorous example:

<p>The computer said <samp>Too much cheese in tray
two</samp> but I didn't know what that meant.</p>