Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[POC] Elm json "source-directories" support #27

Open
wants to merge 23 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 31 additions & 19 deletions data/default.nix
Original file line number Diff line number Diff line change
@@ -1,42 +1,54 @@
{ nixpkgs ? <nixpkgs>
, config ? {}
}:

with (import nixpkgs config);

let
mkDerivation =
{ srcs ? ./elm-srcs.nix
, src
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replaced by srcs

{ elmsrcs ? ./elm-srcs.nix
, srcs
, srcdir ? "src"
, name
, srcdir ? "./src"
, targets ? []
, versionsDat ? ./versions.dat
}:
stdenv.mkDerivation {
inherit name src;
let sanitizePath = str:
let path = builtins.filter (p: p != "..") (lib.splitString "/" str);
in lib.concatStringsSep "/" (map (p: if p == "." then srcdir else p) path);
in stdenv.mkDerivation {
inherit name srcs;
sourceRoot = ".";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be hardcoded - we place generated elm.json with correct paths and the prefix elm make with right dirs.


buildInputs = [ elmPackages.elm ];

patchPhase = let
elmjson = let json = (lib.importJSON ./elm.json) // { source-directories = map sanitizePath srcs; };
in writeText "elm.json" (builtins.toJSON json);
in ''
echo "Generating elm.json"
cp \${elmjson} ./elm.json
cat elm.json
'';

buildPhase = pkgs.elmPackages.fetchElmDeps {
elmPackages = import srcs;
elmPackages = import elmsrcs;
inherit versionsDat;
};

installPhase = let
elmfile = module: "\${srcdir}/\${builtins.replaceStrings ["."] ["/"] module}.elm";
in ''
mkdir -p \$out/share/doc
\${lib.concatStrings (map (module: ''
echo "compiling \${elmfile module}"
elm make \${elmfile module} --output \$out/\${module}.html --docs \$out/share/doc/\${module}.json
'') targets)}
installPhase = ''
mkdir -p $out/share/doc

\${lib.concatStrings (map (module:
let fullmodule = sanitizePath module;
modulename = sanitizePath (lib.removePrefix "./" module);
in ''
echo "compiling \${module}"
elm make \${fullmodule}.elm --output $out/\${modulename}.html --docs $out/share/doc/\${modulename}.json
'') targets)}
'';
};
in mkDerivation {
name = "${name}";
srcs = ./elm-srcs.nix;
src = ./.;
targets = ["Main"];
srcs = ${srcs};
srcdir = "${srcdir}";
targets = ["./Main"];
Copy link
Contributor Author

@turboMaCk turboMaCk May 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BREAKING change: Main.Entry => ./Main/Entry. Supports custom extensions like Main.js.elm or Main.html.elm and entries from different sources (relative path from current dir). UPDATE: elm compiler will no longer support entries like this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we revert this then?

Copy link
Contributor Author

@turboMaCk turboMaCk Sep 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunetely this is still required.

Imagine you have source-directories like this defined in elm.json:

"source-directories": [
  "./src",
  "./src2",
  "../lib"
]

Now when you set targets = [ "Main" ] it's not clear in which directory this main should be located. It can be any one of those.

Another issue is that with current dir. Imagine having this elm.json:

"source-directories": [
   ".",
   "../lib"
]

since I've used srcs attribute in derivation nix will create structure as:

  • $pwd # contains files form ., is named after current durectory
  • lib # contains files from ../lib

this is why Main is prefixed with ./. We need to replace this ./ prefix with the name of directory to which builder placed actual source from current dir.

I wonder if there is any builtin helper that would help to resolve paths in a way nix does it when srcs attribute on mkDerivation is used. I belive this is done in a this shellscript

}
2 changes: 2 additions & 0 deletions elm2nix.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ library
, text
, transformers
, unordered-containers
, vector
, directory
exposed-modules:
Elm2Nix
Elm2Nix.FixedOutput
Expand Down
73 changes: 67 additions & 6 deletions src/Elm2Nix.hs
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,16 @@ import Data.List (intercalate)
import Data.HashMap.Strict (HashMap)
import Data.String.Here
import Data.Text (Text)
import Data.Vector (Vector)
import System.Exit (exitFailure)
import System.IO (hPutStrLn, stderr)

import qualified System.Directory
import qualified Data.HashMap.Strict as HM
import qualified Data.ByteString.Lazy as LBS
import qualified Data.Aeson as Json
import qualified Data.Text as Text
import qualified Data.Vector as Vector

import Elm2Nix.FixedOutput (FixedDerivation(..), prefetch)
import Elm2Nix.PackagesSnapshot (snapshot)
Expand Down Expand Up @@ -67,13 +70,40 @@ parseElmJsonDeps obj =
parseDeps (Object hm) = mapM (uncurry parseDep) (HM.toList hm)
parseDeps v = Left (UnexpectedValue v)

-- TODO: there is likely to be even better abstraction
tryLookup :: HashMap Text Value -> Text -> Either Elm2NixError Value
tryLookup hm key = maybeToRight (KeyNotFound key) (HM.lookup key hm)
where
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reused between parseElmJsonDeps and parseElmJsonSrcs

maybeToRight :: b -> Maybe a -> Either b a
maybeToRight _ (Just x) = Right x
maybeToRight y Nothing = Left y

tryLookup :: HashMap Text Value -> Text -> Either Elm2NixError Value
tryLookup hm key =
maybeToRight (KeyNotFound key) (HM.lookup key hm)
parseElmJsonSrcs :: Value -> Either Elm2NixError (Vector FilePath)
parseElmJsonSrcs obj =
case obj of
Object hm -> do
case tryLookup hm "source-directories" of
-- the `source-directories` part of elm.json is optional
-- if user don't specified this value `src` directory is the default
Left _ -> Right $ pure "./src"
Right srcs ->
case srcs of
Array vec -> mapM extractSrcPath vec
v -> Left $ UnexpectedValue v
v -> Left $ UnexpectedValue v
where
extractSrcPath :: Value -> Either Elm2NixError String
extractSrcPath val =
case val of
String text -> Right (toNixPath (Text.unpack text))
v -> Left $ UnexpectedValue v

toNixPath :: FilePath -> FilePath
toNixPath path =
case path of
"." -> "./."
p@('.':_) -> p
p -> "./" <> p

-- CMDs

Expand All @@ -90,7 +120,17 @@ convert = runCLI $ do
liftIO (putStrLn (generateNixSources sources))

initialize :: IO ()
initialize = runCLI $
initialize = runCLI $ do
liftIO (hPutStrLn stderr "Resolving elm.json source directories into Nix paths...")
res <- liftIO (fmap Json.eitherDecode (LBS.readFile "elm.json"))
elmJson <- either (throwErr . ElmJsonReadError) return res

srcs' <- either throwErr return (parseElmJsonSrcs elmJson)
liftIO (hPutStrLn stderr $ "Using source directories:")
liftIO (mapM (hPutStrLn stderr) srcs')
let srcs = stringifySrcs srcs'
srcdir <- liftIO (getSrcDir srcs')

liftIO (putStrLn [template|data/default.nix|])
where
-- | Converts Package.Name to Nix friendly name
Expand All @@ -102,8 +142,29 @@ initialize = runCLI $
toNixName = Text.replace "/" "-"
name :: String
name = Text.unpack (toNixName baseName <> "-" <> version)
srcdir :: String
srcdir = "./src" -- TODO: get from elm.json
getSrcDir :: Vector FilePath -> IO FilePath
getSrcDir dirs =
case Vector.toList dirs of
[] -> pure "./src" -- in theory redundant
[ dir ] -> pure dir
xs@(h:t) ->
if "./." `elem` xs then do
-- Nix creates dir named after current directory
-- if `srcs` contains `./.`
path <- liftIO System.Directory.getCurrentDirectory
pure (lastDir path)
else
-- We can't really tell which one to choose
-- so we guess it's the first one
pure h
lastDir :: FilePath -> FilePath
lastDir = foldl (\path c -> if c == '/' then "" else path <> [c]) ""
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not the most efficient way to implement this but quite concise

-- TODO: Improve
stringifySrcs :: Vector FilePath -> String
stringifySrcs xs =
"[\n"
<> foldr (\i acc -> " " <> i <> "\n" <> acc) "" xs
<> " ]"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: use QuasiQuotes


-- Utils

Expand Down