Elixir anonymous function shorthand

Elixir’s Getting Started guides go over the &Module.function/arity and &(...) anonymous function shorthand, but there are a couple neat tricks that are not immediately apparent about this shorthand.

For example, you can do something like &"...".

iex> hello_fun = &"Hello, #{&1}"
iex> hello_fun.("Keita")
"Hello, Keita"

Let’s have some more fun.

iex> fun = &~r/hello #{&1}/
iex> fun.("world")
~r/hello world/
iex> fun = &~w(hello #{&1})
iex> fun.("world")
["hello", "world"]
iex> fun.("world moon mars")
["hello", "world", "moon", "mars"]
iex> fun = &if(&1, do: "ok", else: "not ok")
iex> fun.(true)
"ok"
iex> fun.(false)
"not ok"

You can even use defmodule to create an anonymous function that defines a new module.

iex> fun = &defmodule(&1, do: def(hello, do: unquote(&2)))
iex> fun.(Hello, "hello there")
{:module, Hello, <<...>>, {:hello, 0}}
iex> Hello.hello
"hello there"

(Note that I don’t recommend overusing it like I did here! The only one that has been really useful to me was the first example, &"...")

Elixir: A year (and a few months) in

In the beginning of 2015, I wrote a blog post about how my then-current programming language of choice (Ruby) was showing itself to not be as future-proof as I would have liked it to be.

A lot has changed since then, but a lot has remained the same.

First: I have started a few open-source Elixir projects:

  • Exfile — a file upload handling, persistence, and processing library. Extracted from an image upload service I’m working on (also in Elixir).
  • multistream-downloader — a quick tool to monitor and download HTTP-based live streams.
  • runroller — a redirect “unroller” API, see the blog post about it.

The initial push to get me in to Elixir was indeed its performance, but that’s not what kept me. At the same time, I also tried learning Go and more recently, Rust has caught my attention.

In most cases, languages like Go and Rust can push more raw performance out of a simple “Hello World” benchmark test — exactly what I initially did to compare Ruby on Rails to Elixir / Phoenix. The more I used Elixir, the more I gained an appreciation for what I now regard critical language features (yes, most of these apply to Erlang and other functional languages too).

Immutability

This is a big one. Manipulating a mutable data structure is essentially changing the data in memory directly. This, however, is susceptible to the classic race condition bug programmers encounter when writing multithreaded programs:

  • Thread 1 reads “1” from “a”
  • Thread 2 reads “1” from “a”
  • T1 increments “1” to “2” and writes it to “a”
  • T2 increments “1” to “2” and writes it to “a”

In this case, the intended value is “3” because the programmer incremented “a” two times, but the actual value in memory is “2” due to the race condition.

In systems with immutable variables, this class of bug doesn’t exist. It forces the programmer to be explicit about shared state.

Lightweight Processes

A typical Erlang (Elixir) node can have millions of processes running on it. No, these are not your traditional OS processes — they are not threads, either. The Erlang VM uses its own scheduler and thread pool to execute code. The memory overhead of a single process is usually very light — on my machine, it’s 2.6kb (SMP and HiPE enabled).

Inter-Process Communication

Another big one.

Want to send a message to another process?

send(pid, :hello)

Want to handle a message from another process?

receive do
  :hello -> puts "I received :hello"
end

This works on processes that are running in the current node — but it also works across nodes. The syntax is exactly the same. The “pid” variable in the example is able to refer to a process anywhere in the cluster of nodes. The other node doesn’t even have to be Erlang, it just needs to be able to speak the same language, the distribution protocol and the external term format.

OTP

You can’t talk about Erlang or Elixir without bringing up OTP. OTP stands for “Open Telecom Platform” (Erlang was initially developed by Ericsson for use in telecom systems). OTP is a framework with many battle-tested tools to help you build your application — for example,

  • gen_server – an abstraction of the client-server model
  • gen_fsm – a finite state machine
  • supervisor – a supervisor process that automates recovery from temporary failures

OTP is, for all intents and purposes, part of the Erlang standard library. Thus, it is automatically included in any Elixir application as well. The nuts and bolts of OTP are out of the scope of this blog post, but having such a rich toolbox is like a breath of fresh air coming from Ruby (I thought the same when switching full-time from PHP to Ruby).

Conclusion

I’ve learned a lot in this past year, and yet I feel like I’ve only scratched the surface. Thanks for reading!

Playing around with AWS Certificate Manager

I’m a big Let’s Encrypt fan. They provide free SSL certificates for your web servers so you can protect the traffic from prying eyes. In fact, the connection between your web browser and my blog server is made private thanks to Let’s Encrypt.

Using Let’s Encrypt requires some setup and automation on your part if you want to use it in the AWS cloud, but AWS recently launched something called the AWS Certificate Manager or “ACM”. ACM takes care of issuing, renewing, and provisioning certificates for you — which is great because uploading SSL certificates to CloudFront and Elastic Load Balancers is not the most fun thing to do. I would pay for this, but Amazon has decided to give it to everyone for free. ????

As with anything AWS, this has a couple catches, but if you run your cloud resources in AWS you probably won’t be worried about them:

  • You don’t have access to the private key, which means you can’t use the same certificate elsewhere.
  • ACM is currently1 only available in the us-east-1 region, and you can’t use ACM certificates across regions (with the exception of CloudFront, which doesn’t have a region).

So, I decided to test it out on a silly little CloudFront distribution I have running I made for a blog post as a demonstration of how you can use S3 and CloudFront to serve a single-page JavaScript app with “proper” URLs.

Test it out for yourself:

https://single-page-test.kkob.us


  1. as of this writing (2016/2/10)

JavaScript Unit Tests in a Phoenix Application

There’s a guide to writing browser acceptance tests for Phoenix. Acceptance tests are nice, but sometimes you want to have unit tests. This is very easy to do with your Elixir code, but what about your JavaScript code that lives inside your Phoenix application?

I couldn’t find a good guide on this, so I’ll go over what I have set up for one of my latest Phoenix projects.

Setup

First, install mocha if you haven’t already. I’ll be using mocha, but you can use whatever test runner you want to. You’ll also need babel-register — this will allow you to use Babel while the tests are being run in Node.

$ npm install --save-dev mocha
$ npm install --save-dev babel-register

Set up mocha to run your tests in package.json. The default Phoenix & Brunch installation doesn’t include a “scripts” section, so you’ll probably have to create it. If it’s already there, just add the “test” line.

{
  "dependencies": {
    ...
  },
  "scripts": {
    "test": "mocha --compilers js:babel-register test/js/**/*.js"
  }
}

Now, set up Babel to pick up the default preset (if you’re setting a different preset in your brunch-config.js, you just replace es2015 with what you have there). Put this in .babelrc in the root directory of your project.

{
  "presets": ["es2015"]
}

Now, you can put JavaScript unit tests in test/js and by running npm test, they will run inside Node.

Note that npm test will not be automatically run when you run mix test, so you’ll have to change the way tests are run on your CI run by either changing the test command to mix test && npm test or adding the npm test command to your CI configuration.

Example Test

Say we have a module that does nothing but export a function that returns the string "something":

export default function() {
  return "something";
}

This file lives in web/static/js/something.js.

Now, let’s write a test for it in test/js/something_test.js:

import assert from 'assert';
import something from '../../web/static/js/something';

describe('something()', function() {
  it('does something', function () {
    assert.equal('something', something());
  });
});

Now, run the tests:

$ npm test

> @ test /Users/keita/personal/phoenix-mocha-example
> mocha --compilers js:babel-register test/js/**/*.js



  something()
    ✓ does something


  1 passing (8ms)

This example project is available on GitHub if you would like to take a closer look at it. As always, please leave a comment or get in touch if you’d like to provide some input!

Elixir’s StringIO may not be what you think it is

In Ruby, there is a very handy class called StringIO. Basically, it allows you to treat a string like you would an IO object, such as an open file, etc. Very useful for in-memory “files” that you may not want to write to a temporary file.

In Elixir, there is a module called StringIO in the standard library. At first glance, these seem pretty similar:

Ruby:

Pseudo I/O on String object.

Elixir:

This module provides an IO device that wraps a string.

However, there are some subtle differences. For example, you can’t rewind the position of an Elixir StringIO:

iex(1)> {:ok, io} = StringIO.open("foo")
{:ok, #PID<0.59.0>}
iex(2)> IO.read(io, :all)
"foo"
iex(3)> :file.position(io, :bof)
===> hang!

(For more information about the :file.position/2 function, check the docs out. :bof stands for “beginning of file”)

Let’s see why this is happening. StringIO has a function to show its current buffers:

iex(1)> {:ok, io} = StringIO.open("foo")
{:ok, #PID<0.59.0>}
iex(2)> StringIO.contents(io)
{"foo", ""}
iex(3)> IO.read(io, :all)
"foo"
iex(4)> StringIO.contents(io)
{"", ""}

Basically, after you read data from an Elixir StringIO, it’s gone. So, we look to Erlang. Erlang’s file:open/2 function accepts a ram option that we might be interested in:

Returns an fd() which lets the file module operate on the data in-memory as if it is a file.

Let’s try it out.

iex(1)> {:ok, io} = :file.open("foo", [:ram, :binary])
{:ok, {:file_descriptor, :ram_file, #Port<0.1471>}}
iex(2)> IO.binread(io, :all)
"foo"
iex(3)> IO.binread(io, :all)
""
iex(4)> :file.position(io, :bof)
{:ok, 0}
iex(5)> IO.binread(io, :all)
"foo"

Rewinding works now.

Note that because of differences between Elixir’s IO / File modules and Erlang’s file module (probably related to how Elixir works with character encodings), you have to use the binary option to :file.open/2 and the bin-prefixed functions in Elixir land.

Web Development Tools: Mailtrap

This is the inaugural article of my “Web Development Tools” series I plan on continuing for at least a few more posts, sharing some of the essential tools I use for web development every day. When I have the chance to work with new people, we always exchange useful information about the tools and libraries we use. This series is an attempt to organize this information.

Today’s pick: Mailtrap

Once upon a time, I was working on a 2.0 for a client. Major overhaul. The database schema was completely different. So, I wrote a data migration script. Runs fine locally. Then, it became time to import test data to the staging environment.

I had completely forgotten that when creating a new user, a “Welcome to our service, please confirm your account!” e-mail was sent out.

You can imagine what happened next. Thousands of e-mails were being sent while the migration script was running, sending “Welcome!” e-mails to unsuspecting customers — with a link to the staging environment.

This continues to be one of the most embarrassing moments of my career to date. Following this event, I sought out to find something that would make sure this never happened again. I found Mailtrap.

Mailtrap is an SMTP server — an SMTP server that doesn’t forward messages to users. Instead, they’re saved to the Mailtrap mailbox, accessible via a nice web GUI.

Incoming Mailbox
Incoming Mailbox

From there, Mailtrap gives you some nice tools — inspecting the text content if you have a multipart e-mail, viewing HTML source, seeing the raw e-mail.

Raw E-mail View -- headers and all.
Raw E-mail View — headers and all.

When I started using Mailtrap, it just had the most basic feature — catching e-mail. Now, it has a lot of very useful features — shared mailboxes so you can share test e-mails with your team, forwarding so you can forward test e-mails to a real client to see how they look — the list goes on.

Today, I use Mailtrap by default in all of my projects that send any kind of e-mail. It’s very simple to set up — they have instructions for the popular platforms and frameworks (including Sendmail, heh!) — and if your framework isn’t in there, they have the plain old SMTP settings for you to plug in.

Mailtrap is free for one mailbox with up to 50 messages, and plans start from $9.99/monthly. Once the 50 message limit is reached, older messages will be deleted to make room for the new messages. For personal projects, the free tier has been more than enough for my needs.

Disclosure: I am not affiliated with Mailtrap, nor am I receiving any compensation (financial or otherwise) from Mailtrap for writing this article.

Mailtrap is a trademark of Railsware Products, Inc..