Tracks
/
Elm
Elm
/
Exercises
/
GitHup API
GitHup API

GitHup API

Learning Exercise

Introduction

JSON

JSON

JSON (JavaScript Object Notation) is a human readable data and file format commonly used to exchange data, in particular for web applications. As such, it is no surprise that is holds a special place in the Elm language.

Elm provides the core modules Json.Decode and Json.Encode to parse and write JSON data. JSON Decoders are a way to declare what type of data is expected and which data structure the data should map to.

Let's import some aliases and types from the modules:

import Json.Decode as Decode exposing (Decoder, Error)
import Json.Encode as Encode exposing (Value)

Using Decode.decodeString : Decoder a -> String -> Result Error a with a decoder and a JSON string will either succeed if the string confirms to the decoder specifications, or fail otherwise. There is no way of inspecting the raw JSON data when defining the decoder, which could feel unusual coming from imperative languages.

Basic data type decoders

JSON has a small number of core data types:

  • String: encoded with double quotes "hello world"
  • Boolean: either true or false
  • Number: such as 34 or 0.12
  • Null: the absence of data null

Each of these is associated with a Decode function:

Decode.decodeString Decode.string """\"hello\""""
    --> Ok "hello"

Decode.decodeString Decode.string """null"""
    --> Err ...

Decode.decodeString Decode.bool """true"""
    --> Ok True

Decode.decodeString Decode.bool """0"""
    --> Err ...

Decode.decodeString Decode.int """12"""
    --> Ok 12

Decode.decodeString Decode.float """3.14"""
    --> Ok 3.14

Decode.decodeString (Decode.null AnyValue) """null"""
    --> Ok AnyValue

Decode.decodeString (Decode.null AnyValue) """true"""
    --> Err ...

Note that Decode.null lets you decide how to model a null in your program by requiring an arbitrary value as argument (maybe (), Nothing, or anything appropriate to your program).

Combining decoders

JSON also defines two data structures to collect core data types:

  • Array: a collection of other data types [0, null, "none", []]
  • Object: a collection of key-value pairs {"id": 1, "is_admin": false, "more_info": {}}

Decode also offers functions to decode these:

Decode.decodeString (Decode.list Decode.int) """[1, 2, 3]"""
    --> Ok [1, 2, 3]

Decode.decodeString (Decode.list Decode.int) """[1, 2, 3, "not an int"]"""
    --> Err ...

Decode.decodeString (Decode.dict Decode.int) """{ "key1": 17, "key2": 71 }"""
    --> Ok (Dict.fromList [("key1", 17), ("key2", 71)])

Decode.decodeString (Decode.dict Decode.int) """{ "key1": 17, "key2": "seventy-one" }"""
    --> Err ...

These functions Decode.list : Decoder a -> Decoder (List a) and Decode.dict : Decoder a -> Decoder (Dict String a) each expect a decoder as argument, which is expected to decode every single element elements of the data structure. If the argument decoder fails to decode any element of an array or object value, the whole decoder fails, which is a common pattern.

Combining simple decoders into more complex ones is the core idea of the technique. We will introduce several more functions that can be used to build arbitrarily complex real-world data decoders through means of an example.

GeoJSON

For practicing combining basic decoders to parse complex JSON data, let's build decoders for parsing (a subset of) GeoJSON data. GeoJSON is a specialized JSON-based data format used to represent geographic features, such as locations, paths or regions, specified with 2D or 3D geographic coordinates, along with other properties.

Geometry Object

A GeoJSON Geometry Object is an object that contains a type field with a string value (there are 7 possible type values) and a coordinates field with geographic coordinates (collected in a structure that depends on the type). For example,

{
  "type": "Point",
  "coordinates": [127.831, 26.461]
}

or

{
  "type": "LineString",
  "coordinates": [
    [127.831, 26.461],
    [127.829, 26.465],
    [127.829, 26.469],
    [127.83, 26.47]
  ]
}

A Geometry Object could be represented in Elm with the following type

type Geometry
  = Point (List Float)
  | LineString (List (List Float))
  | ...

To probe the value of a certain field, we can use Decode.field : String -> Decoder a -> Decoder a which decodes the value of a specific key of an object.

decodePointCoordinates : Decoder (List Float)
decodePointCoordinates =
    Decode.field "coordinates" (Decode.list Decode.float)

decodeLineStringCoordinates : Decoder (List (List Float))
decodeLineStringCoordinates =
    Decode.field "coordinates" (Decode.list (Decode.list Decode.float))

To transform the value successfully parsed by a decoder, we can use Decode.map : (a -> value) -> Decoder a -> Decoder value, or any of its siblings, from Decode.map2 (a -> b -> value) -> Decoder a -> Decoder b -> Decoder value to Decode.map8.

In this case, we want to decide which coordinate decoder to use depending on the value in the type field, which is a perfect job for Decode.andThen : (a -> Decoder b) -> Decoder a -> Decoder b and its friends Decode.succeed : a -> Decoder a and Decode.fail : String -> Decoder a.

decodeGeometry : Decoder Geometry
decodeGeometry =
    Decode.field "type" Decode.string
        |> Decode.andThen
            (\value ->
                case value of
                    "Point" ->
                        decodePointCoordinates |> Decode.map Point

                    "LineString" ->
                        decodeLineStringCoordinates |> Decode.map LineString

                    _ ->
                        Decode.fail "Geometry not implemented yet, or invalid type value"
            )

Note at this point that we could use Decode.andThen to validate the coordinates (checking that the coordinates come in pairs or triplets and checking that the coordinate values make geographic sense).

Decode.decodeString decodeGeometry """{"type": "Point", "coordinates": [127.831, 26.461]}"""
    --> Ok (Point [127.831, 26.461])

Decode.decodeString decodeGeometry """{"type": "LineString", "coordinates": [127.831, 26.461]}"""
    --> Err ...
Feature Object

A GeoJSON Feature Object represents something spatially bounded: it has a type field with the "Feature" value, a geometry field with a Geometry Object or a null, a properties field with an arbitrary JSON Object or a null, and an optional id field with a JSON string or number value.

{
  "type": "Feature",
  "geometry": null,
  "properties": null
}

or

{
  "type": "Feature",
  "id": "0157",
  "geometry": {
    "type": "Point",
    "coordinates": [127.831, 26.461]
  },
  "properties": {
    "country": "Japan"
  }
}

Let's tackle the id field first. If it exists, it must accommodate strings or numbers, so we could either introduce a custom type for the value, or always save is as a string.

To choose between two possible decoders, we can use Decode.oneOf : List (Decoder a) -> Decoder a.

decodeFeatureId : Decoder String
decodeFeatureId =
    Decode.oneOf
        [ Decode.string
        , Decode.int |> Decode.map String.fromInt
        , Decode.float |> Decode.map String.fromFloat
        ]
        |> Decode.field "id"

Decode.decodeString decodeFeatureId """{"id": "seventeen"}"""
    --> Ok "seventeen"

Decode.decodeString decodeFeatureId """{"id": 17}"""
    --> Ok "17"

Decode.decodeString decodeFeatureId """{"type": "Point"}"""
    --> Er ...

To express that the field is optional, we could use the same technique

decodeMaybeFeatureId : Decoder (Maybe String)
decodeMaybeFeatureId =
    Decode.oneOf
        [ decodeFeatureId |> Decode.map Just
        , Decode.succeed Nothing
        ]

But we could instead use Decode.maybe : Decoder a -> Decoder (Maybe a)

decodeMaybeFeatureId : Decoder (Maybe String)
decodeMaybeFeatureId =
    Decode.maybe decodeFeatureId

Decode.decodeString decodeMaybeFeatureId """{"id": 17}"""
    --> Ok (Just "17")

Decode.decodeString decodeMaybeFeatureId """{"type": "Point"}"""
    --> Ok Nothing

Note that decodeMaybeFeatureId (or any decoder wrapped with Decode.maybe) will always succeed since it can always fallback on Ok Nothing, although a decoder that uses it might still fail

Decode.decodeString (Decode.list decodeMaybeFeatureId) """{"id": 17}"""
    --> Err ...

Next, let's focus on the geometry field, which can either be a Geometry Object or be null. We could use Decode.oneOf once again, but there is a better option: Decode.nullable : Decoder a -> Decoder (Maybe a).

decodeFeatureGeometry : Decoder (Maybe Geometry)
decodeFeatureGeometry =
    Decode.nullable decodeGeometry
        |> Decode.field "geometry"

Decode.decodeString decodeFeatureGeometry """{"geometry": {"type": "Point", "coordinates": [127.831, 26.461]}}"""
    --> Ok (Just (Point [127.831, 26.461]))

Decode.decodeString decodeFeatureGeometry """{"geometry": null}"""
    --> Ok Nothing

Decode.decodeString decodeFeatureGeometry """{"geometry": {}}"""
    --> Err ...

Decode.maybe and Decode.nullable have the same signature, but their behavior is different, since the OK Nothing result of a Decode.nullable will only come from the specific JSON value null.

Finally, the properties field can contain an arbitrary JSON Object or a null, which is a job for Decode.value : Decoder Value.

decodeProperties : Decoder (Maybe (Dict String Value))
decodeProperties =
    Decode.nullable (Decode.dict Decode.value)
        |> Decode.field "properties"


Decode.decodeString decodeProperties """{"properties": null}"""
    --> Ok Nothing

Decode.decodeString decodeProperties """{"properties": {"country": "Japan"}}"""
    --> Ok (Just (Dict.fromList [("country", <internals>)]))

Decode.decodeString decodeProperties """{"properties": 17}"""
    --> Err ...

Note that Value is an opaque type, which means you cannot easily probe its content. To use it, you can either write a decoder for it and run it with Decode.decodeValue : Decoder a -> Value -> Result Error a, or send it to JavaScript directly via a port.

We are ready to parse Feature Objects:

type alias Feature =
    { id : Maybe String
    , geometry : Maybe Geometry
    , properties : Maybe (Dict String Value)
    }

decodeFeature : Decoder Feature
decodeFeature =
    Decode.field "type" Decode.string
        |> Decode.andThen
            (\value ->
                if value /= "Feature" then
                    Decode.fail "not a Feature"

                else
                    Decode.map3 Feature
                        decodeMaybeFeatureId
                        decodeFeatureGeometry
                        decodeProperties
            )

Decode.decodeString decodeFeature """{"type": "Feature", "geometry": null, "properties": null}"""
    --> Ok { geometry = Nothing, id = Nothing, properties = Nothing }

"""
{
  "type": "Feature",
  "id": "0157",
  "geometry": {
    "type": "Point",
    "coordinates": [127.831, 26.461]
  },
  "properties": {
    "country": "Japan"
  }
}
"""
  |> Decode.decodeString decodeFeature
    --> Ok { geometry = Just (Point [127.831,26.461]), id = Just "0157", properties = Just (Dict.fromList [("country",<internals>)]) }

JSON Encoders

Encoders allow to write valid JSON from Elm values using the Encode.encode : Int -> Value -> String function. The first argument specifies the amount of indentation in the final result, and the second argument is the JSON value to write.

A Value can either be obtained from Decode.value or be produced from one of the encoders:

Encode.encode 0 (Encode.string "hello")
    --> "hello"

Encode.encode 0 (Encode.bool True)
    --> "true"

Encode.encode 0 (Encode.int 12)
    --> "12"

Encode.encode 0 (Encode.float 3.14)
    --> "3.14"

Encode.encode 0 Encode.null
    --> "null"

Encode.encode 0 (Encode.list Encode.int [1, 2, 3])
    --> "[1,2,3]"

Encode.encode 4 (Encode.list Encode.int [1, 2, 3])
    --> "[\n    1,\n    2,\n    3\n]"

Encode.encode 0 (Encode.dict String.toLower Encode.int (Dict.fromList [("KEY1", 17), ("KEY2", 71)]))
    --> "{\"key1\":17,\"key2\":71}"

Encode.encode 4 (Encode.dict String.toLower Encode.int (Dict.fromList [("KEY1", 17), ("KEY2", 71)]))
    --> "{\n    \"key1\": 17,\n    \"key2\": 71\n}"

as well as with Encode.object : List ( String, Value ) -> Value

Encode.object
    [ ( "key1", Encode.int 17 )
    , ( "key2", Encode.int 71 )
    ]
    |> Encode.encode 0
    --> "{\"key1\":17,\"key2\":71}"

Le's define encoders for the GeoJSON decoders defined earlier

encodeGeometry : Geometry -> Value
encodeGeometry geometry =
    case geometry of
        Point coord ->
            Encode.object
                [ ( "type", Encode.string "Point" )
                , ( "coordinates", Encode.list Encode.float coord )
                ]

        LineString coord ->
            Encode.object
                [ ( "type", Encode.string "LineString" )
                , ( "coordinates", Encode.list (Encode.list Encode.float) coord )
                ]

encodeFeature : Feature -> Value
encodeFeature feature =
    let
        maybeId =
            case feature.id of
                Nothing ->
                    []

                Just id ->
                    [ ( "id", Encode.string id ) ]
    in
    Encode.object
        (maybeId
            ++ [ ( "type", Encode.string "Feature" )
               , ( "geometry"
                 , case feature.geometry of
                    Nothing ->
                        Encode.null

                    Just geometry ->
                        encodeGeometry geometry
                 )
               , ( "properties"
                 , case feature.properties of
                    Nothing ->
                        Encode.null

                    Just dict ->
                        Encode.dict identity identity dict
                 )
               ]
        )

In general, a good test for your decoder-encoder pairs is checking that elmValue |> encoder |> decoder == elmValue.

Instructions

The more you learn about programming, the more you start thinking that this might be more than just a fad. There might even be some business opportunities: people write code, they want to save it, share it, collaborate, maybe get some feedback even.

Alright, let's do it, let's build a service to encourage people to code: hup, hup, people, start using git! Let's call it... GitHup.

You get started on building a REST JSON API: users, pull requests, comments, the backend is swiftly done. Here is a example of the payload being served when you ask the server for the list of comments on a pull request:

[
  {
    "id": 256,
    "pull_request_review_id": 42,
    "user": {
      "id": 101,
      "login": "octodog",
      "avatar_url": "https://githup.com/images/error/octodog_happy.gif",
      "site_admin": false
    },
    "body": "Great stuff!",
    "side": "RIGHT",
    "_links": {
      "self": {
        "href": "https://api.githup.com/repos/octodog/Hello-World/pulls/comments/1"
      },
      "html": {
        "href": "https://githup.com/octodog/Hello-World/pull/1#discussion-diff-1"
      },
      "pull_request": {
        "href": "https://api.githup.com/repos/octodog/Hello-World/pulls/1"
      }
    }
  },
  {
    "id": 11,
    "pull_request_review_id": null,
    "user": {
      "id": 2,
      "login": "hexacat",
      "name": "Alex Kate",
      "avatar_url": "https://githup.com/images/error/hexacat_happy.gif",
      "site_admin": true
    },
    "body": "Amazing stuff!",
    "side": "LEFT",
    "_links": {
      "self": {
        "href": "https://api.githup.com/repos/octodog/Hello-World/pulls/comments/1"
      }
    }
  }
]

Of course, the frontend will be done in Elm, but first you need to figure out what those notorious decoders/encoders are all about.

Note

Humorous story aside, the exercise requirements are taken directly from GitHub's REST API endpoints for pull request review comments. Some field were removed from the JSON schemas for the sake of simplicity, but this is as close to real-world applications as it gets.

1. ID, please!

Your eagle eye has noticed that both comments and users have an id field, perfect place to start.

Define decodeId so that it can decode a JSON object, with an integer id.

Decode.decodeString decodeId """{id: 10}"""
    --> Ok 10

2. What's my name again?

You notice in the example above that not all users have a name.

Define decodeName so that it can decode a string in the name field if there is one. No name? No problem, simply return Nothing. This decoder should never fail.

Decode.decodeString decodeName """{"id": 10, "name": "Otto"}"""
    --> Ok (Just Otto)

Decode.decodeString decodeName """null"""
    --> Ok Nothing

3. User? I hardly know her!

We are pretty much halfway there for decoding users, let's get it done.

Define decodeUser so that it can decode a JSON user object. Make sure to use decodeId and decodeName in decodeUser.

Decode.decodeString decodeUser
    """
    {
      "id": 101,
      "login": "octodog",
      "avatar_url": "https://githup.com/images/error/octodog_happy.gif",
      "site_admin": false
    }
    """
    --> Ok
    -->   { id = 101
    -->   , name = Nothing
    -->   , login = "octodog"
    -->   , avatarUrl = "https://githup.com/images/error/octodog_happy.gif
    -->   , siteAdmin = False
    -->   }

4. I'm a strong independent comment

Some comments in a pull review are standalone, and therefore do not have a pull_request_review_id. However, the specification mention that in such a case, there should still be a pull_request_review_id field, but the value should be null, the poor man's Nothing.

Define decodePullRequestReviewId so that it can decode an integer in the pull_request_review_id field if there is one. If there is no integer, there should be a null, if there is no null the decoder should fail.

Decode.decodeString decodePullRequestReviewId """{"pull_request_review_id": 3}"""
    --> Ok (Just 3)

Decode.decodeString decodePullRequestReviewId """{"pull_request_review_id": null}"""
    --> Ok Nothing

Decode.decodeString decodePullRequestReviewId """{"id": 3}"""
    --> Err ...

5. Side hustle

Next is this interesting looking side field. In a pull review, you can comment on code that was removed (on the Left side of the screen) or on code that was added (on the Right side of the screen).

Define decodeSide so that it can decode either "LEFT" or "RIGHT" in the side field and match it to its corresponding type variant.

Decode.decodeString decodeSide """{"side": "LEFT"}"""
    --> Ok Left

Decode.decodeString decodeSide """{"side": "middle?"}"""
    --> Err ...

Each comment has a set of links, which doesn't seem to be consistent across comments.

Define decodeLinks so that it can decode a collection of URLs in the _links field. The names and number of links can vary, make sure you can account for that. The links themselves are JSON objects that should always have a href field.

Decode.decodeString decodeLinks """
    {
      "_links": {
        "self": { "href": "https://api.githup.com/repos/octodog/Hello-World/pulls/comments/1" },
        "html": { "href": "https://githup.com/octodog/Hello-World/pull/1#discussion-diff-1" },
        "pull_request": { "href": "https://api.githup.com/repos/octodog/Hello-World/pulls/1" }
      }
    }
    """
    --> Ok
    -->   (Dict.fromList
    -->       [ ( "self", "https://api.githup.com/repos/octodog/Hello-World/pulls/comments/1" )
    -->       , ( "html", "https://githup.com/octodog/Hello-World/pull/1#discussion-diff-1" )
    -->       , ( "pull_request", "https://api.githup.com/repos/octodog/Hello-World/pulls/1" )
    -->       ]
    -->   )

7. Yes comments

Almost there, let's get it done.

Define decodeComment and decodeComments so that they can decode a single JSON comment object or a list of comments respectively. Make sure to use all of the functions defined so far in decodeComment, with the exception of decodeName. Of course, decodeComments should use decodeComment as well.

8. It all ties together

There is one more feature missing in all of this: being able to send a new comment to the server. For this, you need achieve the opposite of a decoder, you need to be able to transform an Elm Comment into a JSON Value.

Define encodeComment so that it can encode any valid Comment. The order of the fields doesn't matter, as long as the encoder can produce a value that decodeComment can decode.

Edit via GitHub The link opens in a new window or tab
Elm Exercism

Ready to start GitHup API?

Sign up to Exercism to learn and master Elm with 23 concepts, 94 exercises, and real human mentoring, all for free.