Gatlin.io Icon
Menu Icon
Image of Austin Gatlin's face
May 22, 2021

On an Ecto Preload Dilemma

In this short blog post I'm going to defend the position that you should try to avoid using Ecto.Repo.preload/3. I am then going to propose an approach utilizing secondary contexts and the expression/pipe syntax of Ecto.Query.preload/3.

Here's an example of the type of code snippet I will be arguing for by the end of this article.

defmodule Twitter.Social.Posts do
  alias Twitter.Social.Post
  import Ecto.Query

  def get!(id, preloads \\ []) do
    Post |> preload(^preloads) |> Repo.get!(id)
  end
end

The Dilemma

Ecto (rightfully so) does not load associations by default. Loading associations is the developer's responsibility and is done via either Ecto.Repo.preload/3 or Ecto.Query.preload. When I was first investigating this, Ecto.Query seemed to be the inappropriate option because it was at a "lower level" in that it was essentially a member of the ORM API. Thus, I began regularly using Repo.preload whenever I needed to load an association.

There are a few issues to be aware of when using Repo.preload. First, it's hitting the database to fetch the association data. That is a round-trip to the database and back. Second, to use Repo.preload means you already have the primary data to being with and are only missing the particular associations you need. The implication here is that when you originally fetched your primary data, you didn't fetch all the data you needed. While I'm sure there are many places in a codebase where Repo.preload calls are justified, I bet that, more often than not, Repo.preload is a "code smell" and really you should go back to wherever your original data fetched the data and update it to fetch all the data you need. The third, and last point, is a bit abstract, and also might just be an elaboration on my second point above. I think that regular use of Repo.preload might be an indication that you aren't confident in the shape of the data moving through your codebase. Each time you use Repo.preload, ask yourself "should I already have this association data by this point in the codepath?" Speaking for myself in the least, I realized that when I used Repo.preload, I was essentially addressing the "symptom" of not having a particular association, instead of addressing the "illness" in my codebase of not having a good, systemic way to fetch the data I needed.

The Solution

The first part of the solution is utilizing the concept of Devon Estes's secondary context. Without going into too much detail (I recommend reading his blog post), secondary contexts help isolate where preload functionality lives. Now, even if you have multiple preload calls, they are at least only within the secondary context file for a given schema, which will make refactoring easier later.

The second part of the solution involves the fact that Ecto.Query supports two different syntaxes. The more common "keyword syntax", and the less common macro/expression/pipe syntax that is designed for usage with the pipe operator. The last example in the Ecto.Query.preload/3 documentation shows how you can chain a Query.preload/3 onto an otherwise ordinary Query or Repo call. The power of this is in the fact that you can construct a query that knows it needs particular associations within a schema before you even know which row in the database you want to request.

The final piece of the puzzle is picking which function in your secondary context to extend with preloads. I choose get!, but obviously this will be at your discretion. Here is the final result:

defmodule Twitter.Social.Posts do
  alias Twitter.Social.Post
  import Ecto.Query

  def get!(id, preloads \\ []) do
    Post |> preload(^preloads) |> Repo.get!(id)
  end
end

Note that this function supports complex association requests given that it's simply a passthrough to the underlying preload API. Now you can write things like:

post = Posts.get!(3, [comments: :author])
post.comments |> Enum.at(0) |> &(&1.author.name).()

Conclusion

I wanted to share this approach because I thought it was an interesting solution to the preload dilemma I was experiencing. I hope this can help others that might be in a similar situation. Thanks for reading!