[POC] Elm json "source-directories" support #27

turboMaCk · 2019-05-19T19:46:10Z

~~At this moment this is only proof of concept that needs to be discussed and polished.~~

Related to #6 and #19

This is an attempt to implement source-directories (elm.json) compatibility for both code generation and nix derivations.

Background

Elm make performs lookups for modules based on optional source-directories attr in elm.json file. If this attribute is not present ./src is a default destination where it looks for .elm files.

it's possible to import foo/A.elm within bar/B.elm as import Aand import B if both foo and bar are defined in source-directories
it's possible to do elm make foo/A.elm bar/B.elm as well (as an entry).
it's possible to use any relative path like ../../baz

Implementation

Because of this, we need to use srcs instead of src. srcs attribute is currently generated in Haskell from elm.json. Paths like ../foo are also valid and needs to be supported. In the case of . (current directory) nix seems to be using the name of the current directory. This means that ["../foo", "."] within app results in 2 directories foo app. Because of changes in directories structure, new elm.json must be produced during the build - done via nix.

Details will be in source comments.

.gitignore

shell.nix

turboMaCk

commented not so obvious parts of the diff.

data/default.nix

turboMaCk · 2019-05-19T19:49:45Z

data/default.nix

@@ -6,27 +6,47 @@ with (import nixpkgs config);

 let
  mkDerivation =
-    { srcs ? ./elm-srcs.nix
-    , src


replaced by srcs

turboMaCk · 2019-05-19T19:51:29Z

data/default.nix

    , targets ? []
    , versionsDat ? ./versions.dat
    }:
    stdenv.mkDerivation {
-      inherit name src;
+      inherit name srcs;
+      sourceRoot = ".";


this can be hardcoded - we place generated elm.json with correct paths and the prefix elm make with right dirs.

data/default.nix

turboMaCk · 2019-05-19T19:57:43Z

src/Elm2Nix.hs

+-- TODO: there is likely to be even better abstraction
+tryLookup :: HashMap Text Value -> Text -> Either Elm2NixError Value
+tryLookup hm key = maybeToRight (KeyNotFound key) (HM.lookup key hm)
+  where


reused between parseElmJsonDeps and parseElmJsonSrcs

turboMaCk · 2019-05-25T12:36:03Z

data/default.nix

  srcdir = "${srcdir}";
+  targets = ["./Main"];


BREAKING change: Main.Entry => ./Main/Entry. Supports custom extensions like ~~Main.js.elm or Main.html.elm and entries from different sources (relative path from current dir).~~ UPDATE: elm compiler will no longer support entries like this.

Can we revert this then?

unfortunetely this is still required.

Imagine you have source-directories like this defined in elm.json:

"source-directories": [ "./src", "./src2", "../lib" ]

Now when you set targets = [ "Main" ] it's not clear in which directory this main should be located. It can be any one of those.

Another issue is that with current dir. Imagine having this elm.json:

"source-directories": [ ".", "../lib" ]

since I've used srcs attribute in derivation nix will create structure as:

$pwd # contains files form ., is named after current durectory

lib # contains files from ../lib

this is why Main is prefixed with ./. We need to replace this ./ prefix with the name of directory to which builder placed actual source from current dir.

I wonder if there is any builtin helper that would help to resolve paths in a way nix does it when srcs attribute on mkDerivation is used. I belive this is done in a this shellscript

turboMaCk · 2019-05-25T15:09:44Z

src/Elm2Nix.hs

+          pure (Vector.head dirs)
+
+    lastDir :: FilePath -> FilePath
+    lastDir = foldl (\path c -> if c == '/' then "" else path <> [c]) ""


not the most efficient way to implement this but quite concise

turboMaCk · 2019-05-25T15:42:37Z

@domenkozar This is ready for a review.

principles followed:

as simple and concise as I managed to make it
pkgs.mkDerivation of default.nix doesn't contain any hs generated code - all project specific tab is argument defined in in block
as much breaking changes as possible

Some of the functions are not exactly most optimal or robust (path handling mostly).

turboMaCk · 2019-05-25T17:11:20Z

test/source-directories/a-src/elm.json

+    "type": "application",
+    "source-directories": [
+        ".",
+        "../b-src/src"


turboMaCk · 2019-05-25T17:11:42Z

test/source-directories/a-src/elm.json

+{
+    "type": "application",
+    "source-directories": [
+        ".",


turboMaCk · 2019-05-27T21:10:30Z

src/Elm2Nix.hs

+    stringifySrcs xs =
+      "[\n"
+      <> foldr (\i acc -> "    " <> i <> "\n" <> acc) "" xs
+      <> "  ]"


todo: use QuasiQuotes

domenkozar · 2019-05-30T09:22:26Z

Sorry for taking this long, I've just moved over the continent and don't have immediate time to review this work. Hope you're ok waiting a few more days or so.

turboMaCk · 2019-05-30T12:51:42Z

no worries. I'm busy as well. Take your time, there is no need to rush anything. Given the nature of these changes, I expect this to take maybe even a few weeks still

turboMaCk · 2019-09-29T12:02:00Z

This is how it works on a project with rather complicated source-directories setting in elm.json:

That's being said this is still not good enough for merge.

damien-biasotto · 2021-08-17T01:03:07Z

What is the status of this pull request?

Also, correct me if I'm wrong, but once merged could it help us import private elm packages?

turboMaCk · 2021-08-17T06:30:03Z

I'm not sure how you emulate private packages but if you use source-directories then yes this would solve it. But the PR is on freeze. There are few hacks that made this work and it would probably require some more discussion before being really ready. Other than that I think this PR might still work so you can try to just run elm2nix version from this.

turboMaCk commented May 19, 2019

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

turboMaCk commented May 19, 2019

View reviewed changes

shell.nix Outdated Show resolved Hide resolved

turboMaCk commented May 19, 2019

View reviewed changes

turboMaCk added 7 commits May 23, 2019 18:09

Haskell parsing of SRC dirs & base nix support

05901cd

Generate custom elm.json via nix

bec0af0

add todo comments

9f0e44e

additional TODO

d293a82

one more TODO

aaebde0

srcdir detection

505750c

fix toNixPath

f7b8cc8

turboMaCk force-pushed the better-support-for-source-dirs branch from ce64e0f to f7b8cc8 Compare May 23, 2019 16:10

turboMaCk added 9 commits May 23, 2019 18:25

move dirty patches to patchPhase

256138e

improvements

58a2a19

fix targets in src

1c7ab69

simplify source-directories update

b562bee

improve output file names

bb5bcc1

cleanup default.nix

b65e410

fix syntax err

e1c290e

fix syntax err

7d67711

move pathPhase above buildPhase

571cac6

turboMaCk commented May 25, 2019

View reviewed changes

unify codestyle

2f46543

turboMaCk commented May 25, 2019

View reviewed changes

turboMaCk changed the title ~~POC "source-directories" support~~ Elm json "source-directories" support May 25, 2019

turboMaCk added 2 commits May 25, 2019 19:08

add test

9bd289e

gitignore stuff generated during tests

7786955

turboMaCk commented May 25, 2019

View reviewed changes

test/source-directories/a-src/elm.json

"type": "application",

"source-directories": [

".",

"../b-src/src"

Copy link

Contributor Author

turboMaCk May 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests #6

turboMaCk commented May 25, 2019

View reviewed changes

test/source-directories/a-src/elm.json

{

"type": "application",

"source-directories": [

".",

Copy link

Contributor Author

turboMaCk May 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests #19

turboMaCk commented May 27, 2019

View reviewed changes

turboMaCk added 2 commits September 22, 2019 15:16

Merge branch 'master' into better-support-for-source-dirs

0b86e5d

Merge branch 'master' into better-support-for-source-dirs

0a26a19

bring elmsrcs back (previously srcs)

56be5d7

turboMaCk changed the title ~~Elm json "source-directories" support~~ [POC] Elm json "source-directories" support Sep 29, 2019

remove lib test

f9511bd

domenkozar force-pushed the master branch 5 times, most recently from c9c7ad2 to 81bfde0 Compare September 6, 2022 18:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[POC] Elm json "source-directories" support #27

[POC] Elm json "source-directories" support #27

turboMaCk commented May 19, 2019 •

edited

Loading

turboMaCk left a comment

turboMaCk May 19, 2019

turboMaCk May 19, 2019

turboMaCk May 19, 2019

turboMaCk May 25, 2019 •

edited

Loading

domenkozar Sep 29, 2019

turboMaCk Sep 29, 2019 •

edited

Loading

turboMaCk May 25, 2019

turboMaCk commented May 25, 2019

turboMaCk May 25, 2019

turboMaCk May 25, 2019

turboMaCk May 27, 2019

domenkozar commented May 30, 2019

turboMaCk commented May 30, 2019

turboMaCk commented Sep 29, 2019 •

edited

Loading

damien-biasotto commented Aug 17, 2021

turboMaCk commented Aug 17, 2021

[POC] Elm json "source-directories" support #27

Are you sure you want to change the base?

[POC] Elm json "source-directories" support #27

Conversation

turboMaCk commented May 19, 2019 • edited Loading

Background

Implementation

turboMaCk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turboMaCk May 25, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turboMaCk Sep 29, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

turboMaCk commented May 25, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenkozar commented May 30, 2019

turboMaCk commented May 30, 2019

turboMaCk commented Sep 29, 2019 • edited Loading

damien-biasotto commented Aug 17, 2021

turboMaCk commented Aug 17, 2021

turboMaCk commented May 19, 2019 •

edited

Loading

turboMaCk May 25, 2019 •

edited

Loading

turboMaCk Sep 29, 2019 •

edited

Loading

turboMaCk commented Sep 29, 2019 •

edited

Loading