An example of nested forms, nested changesets, and error handling

It took me some time to get the server-side code behind the following form working. If you want to do something similar, this post may be of use.

I welcome comments and would be happy to rewrite my code and this example based on them.

There are two Postgres tables (and, so, Ecto schemas) involved. Animal has many ServiceGaps. An animal with its loaded associations looks like something like this:

%Crit.Setup.Schemas.Animal{
  id: 1,
  name: "Hank",
  service_gaps: [
    %Crit.Setup.Schemas.ServiceGap{
      animal_id: 1,
      id: 1,
      reason: "seed",
      span: %Ecto.Datespan{
        first: ~D[2023-01-01],
        last: ~D[2023-03-03],
        lower_inclusive: true,
        upper_inclusive: false
      }
    }
  ],
}

I'm going to ignore how the Datespan works for this post and just deal with the first two levels of the tree.

A nice feature of Ecto is that you can give Ecto.update a tree of changesets and it will figure out the appropriate set of SQL UPDATE and INSERT commands to send to the database.  Here's such a nested Changeset:

#Ecto.Changeset<
  action: nil,
  changes: %{
    service_gaps: [
      #Ecto.Changeset<
        action: :insert,
        changes: %{...},
        errors: [...],
        data: #Crit.Setup.Schemas.ServiceGap<>,
        valid?: false
      >,
      #Ecto.Changeset<
        action: :update,
        changes: %{...}
        errors: [],
        data: #Crit.Setup.Schemas.ServiceGap<>,
        valid?: true
      >
    ]
  },
  valid?: false
>

Dealing with error handling, though, is a bit tricky. My test file, update_test.exs, is a list of  special cases. You can use them as checklist when writing your own code.

The rest of the post shows how I handled those cases.

The new ServiceGap

The first interesting case is the subform that creates a new service gap:

Here's what's required:

1. The fully-loaded Animal's list of ServiceGaps has nothing corresponding to the empty form. It has to be added specially.
2. The user can fill out the subform wrongly. For example, each of the fields is required. Such mistakes have to be reported back.
3. However, there should be no errors if the user doesn't even try to fill out the form. The case where only the "Back on" field is blank is different than when it and all the others are blank.
4. If the whole Animal form is shown again (because of errors somewhere within it), errors outside of the empty subform should not affect its display. It should be present and correctly annotated with its own errors, if any. (Spoiler: this doesn't happen automatically.)

Now, any nested form has this structure:

<%= form_for @changeset, ..., [], fn f -> %>
  ...
  <%= inputs_for(f, :service_gaps, fn gap_f -> %>
    ...
  <% end %>
<% end %>

An easy way to insert a blank empty form is to use the :prepend option to inputs_for:

<%= inputs_for(f, :service_gaps, [prepend: %ServiceGap{}], fn gap_f -> %>

I've chosen not to do that. I don't have an exceptionally good reason. As far as I can tell, though, the above requirements don't allow inputs_for to be the only case where a ServiceGap list is adjusted. No matter what, it has to be done in two different places. So I don't think using :prepend greatly simplifies things.

Instead, I have the code that produces a changeset for the Animal form add the empty ServiceGap:

  def form_changeset(animal) do
    animal
    |> prepend_empty_service_gap
    |> change
  end

  def prepend_empty_service_gap(animal) do
    Map.put(animal, :service_gaps, 
            [%ServiceGap{} | animal.service_gaps])
  end

Handling an empty new ServiceGap

The code that handles an Animal update first creates a changeset. In that process, it uses cast_assoc to also produce ServiceGap changesets:

    struct
    |> cast(attrs, required)
    |> validate_required(required)
    |> ToSpan.synthesize(attrs)
    |> cast_assoc(:service_gaps)        # <====
    |> constraint_on_name()
    |> optimistic_lock(:lock_version)

If the prepended originally-empty subform is still empty, we have to handle its data specially. We don't want the user to see complaints about missing required fields. One way to do that is to have ServiceGap.changeset notice the special case and mark the resulting changeset with the :ignore action. However, doing that naively will cause the user to see a sensible initial form...

... turn into a bewildering form after an error:

Note that "Edit or delete earlier gaps" has disappeared. The values that were under it are now displayed under "Add a gap". What's happened is that the data for the empty form has disappeared and the code isn't smart enough to realize it.

The underlying problem is that cast_assoc throws away any changeset marked :ignore. We have to add it back.

Rather than use :ignore, I chose to remove empty subform data from the attrs before the Animal changeset is constructed:

  def update_changeset(struct, given_attrs) do
    attrs =
      Common.filter_out_unstarted_subforms(
        given_attrs,
        "service_gaps",
        ServiceGap.unstarted_form_sentinels)
    required = [:name, :lock_version]
    struct
    |> cast(attrs, required)  # The code you saw earlier
    ...

Seems ugly, but the code to detect when to :ignore doesn't seem better, and I have to add back an empty subform no matter what.

Error adjustments

Now, suppose there's an error: either in the Animal fields, in the fields for a new ServiceGap, or in the fields used to update an existing ServiceGap. Sadly, there are three cases to handle. I'll show the whole function first, then explain it. Not shown is code used to handle an optimistic locking error (using Changeset.optimistic_lock). That's very different in that the new version of the Animal has to be loaded and sent back to be displayed in a fresh form.

Before the code, it's useful to review how cast_assoc treats subform data.

1. If none of the subforms have been edited, the changeset for the outer form will have no changes for the associated "has many" field. In this case, that means that Changeset.fetch_change(changeset, :service_gap) will return :error, indicating the user intends no changes to any ServiceGap.
2. If any of the subforms have been edited, the outer changeset will include changesets for all of the subforms. (Ecto will not issue an SQL command for any of the ones without their own changes, but they will exist.)

defp changeset_for_other_error(changeset) do
  case Changeset.fetch_change(changeset, :service_gaps) do
    {:ok, [ %{action: :insert} | _rest ]} ->
      changeset

    {:ok, only_has_service_gap_updates} ->
       Changeset.put_change(
         changeset, 
         :service_gaps,
         [Changeset.change(%ServiceGap{}) 
            | only_has_service_gap_updates
         ])

    :error -> 
       %{changeset 
          | data: Animal.prepend_empty_service_gap(changeset.data)}
    end
  end

There are three cases to handle.

Case 1: The user tried to make a new ServiceGap and failed

Because of that, the POST parameters did not have the values for the new ServiceGap removed by Animal.update_changeset: they were converted to a ServiceGap changeset. Because there was that one ServiceGap changeset, all the existing ServiceGaps were also turned into changesets. Therefore, the Animal's changeset-with-errors needn't be changed before it is used to populate the form with edited values and error messages (definitely including the ones for the failed attempt, possibly including ones for existing ServiceGaps).

defp changeset_for_other_error(changeset) do
  case Changeset.fetch_change(changeset, :service_gaps) do
    {:ok, [ %{action: :insert} | _rest ]} ->
      changeset

Case 2: No new ServiceGap, but changes were attempted in old ServiceGaps

This case is like the above, except that the completely-blank subform was removed before a changeset was created. To get that form added back when the overall Animal form is created, an empty ServiceGap changeset has to be prepended to the :service_gaps list.

{:ok, only_has_service_gap_updates} ->
  Changeset.put_change(
    changeset, 
    :service_gaps,
    [Changeset.change(%ServiceGap{}) | only_has_service_gap_updates])

Case 3: User didn't edit any ServiceGap

In this case, the error was in the enclosing Animal field. Because the user didn't edit any ServiceGap subform, there are no changes for the Animal's :service_gaps field. If the Animal's changeset were given to the edit-form's template, the blank ServiceGap field would not be added. So it has to be added explicitly. Since there are no ServiceGap changesets, it's easiest to add the empty data to the Animal changeset's underlying data:

:error -> 
   %{changeset | data: 
       Animal.prepend_empty_service_gap(changeset.data)}

Note that the same trickery would be required if the :prepend option were given to inputs_for. That's because inputs_for does not obey :prepend if the changeset has an error. That is, the fact that the enclosing Animal changeset has :valid set to false prevents the enclosed inputs_for from adding an empty subform.

Postscript

It bothers me that so much knowledge about how to handle forms gets pushed down close to the layout of Ecto schemas, which seem like they should only be about the business domain.

But every time I try to migrate form-handling considerations closer to the crit_web controller layer, I get into problems. It seems like that is cutting against the grain of the Ecto framework. I would very much like to see counterexamples.

Code

Here is the commit corresponding to this version of the writeup: https://github.com/marick/crit19/tree/nested-form-post Because a lot of flailing went into getting all this to work, the code is kind of embarrassing. It will get better over time.

Relevant test cases. Note these tests check what appears in the HTML output. Given the variety of changes that are sometimes required, the real test has to be what the user sees.

The partial that shows an edit form, with help from view functions, including one that puts headers between the "add a new" and "change existing" sections of the form - in a way that seems it must be improvable.

The code that creates animal changesets and service gap changesets.

The code that performs the update and handles the resulting errors.