New Interface (#71)

* Migrate to picocli

* Adding basic Repl

* Adding support for repl

* Adding sub commands for spells

* Using migrate as the name of the sub-command

* chore: nested command help renderer (#73)

---------

Co-authored-by: Manik Magar <manik.magar@gmail.com>

Author: machaval

README.md

@@ -2,7 +2,7 @@
 
 **DataWeave CLI** is a command-line interface that allows `querying`, `filtering`, and `mapping` structured data from different data sources like `JSON`, `XML`, `CSV`, `YML` to other data formats. It also allows to easily create data in such formats, all through the DataWeave language. For example:
 
-`dw 'output json --- { message: ["Hello", "world"] joinBy " "}'`
+`dw run 'output json --- { message: ["Hello", "world"] joinBy " "}'`
 
 The DataWeave language is in the process of being open-sourced. You can read our announcement [here](https://blogs.mulesoft.com/news/dataweave/). Our journey has just begun and it will take some time for the code to be available. In the meantime, we want to start engaging with our community to understand how DataWeave could be used and integrated. 
 
@@ -81,75 +81,29 @@ If it is not, go to the `bin` directory referenced in the installation instructi
 The following example shows the DataWeave CLI documentation
 
 ```bash
-dw --help
+dw help
 ```
 
 ```bash
-usage: dw [--eval] [-f <file-path>] [--help] [-i <input-name input-path>]
-       [--migrate <dw1-file-path>] [-o <output-path>] [-p <param-name
-       param-value>] [--privileges <privileges>] [--silent]
-       [--untrusted-code] [-v] [--version] [--add-wizard <wizard-name>]
-       [--list-spells] [--local-spell <spell-folder>] [--new-spell
-       <spell-name>] [-s <spell-name>] [--update-grimoires]
-
-
-.........................................................................
-.%%%%%....%%%%...%%%%%%...%%%%...%%...%%..%%%%%%...%%%%...%%..%%..%%%%%%.
-.%%..%%..%%..%%....%%....%%..%%..%%...%%..%%......%%..%%..%%..%%..%%.....
-.%%..%%..%%%%%%....%%....%%%%%%..%%.%.%%..%%%%....%%%%%%..%%..%%..%%%%...
-.%%..%%..%%..%%....%%....%%..%%..%%%%%%%..%%......%%..%%...%%%%...%%.....
-.%%%%%...%%..%%....%%....%%..%%...%%.%%...%%%%%%..%%..%%....%%....%%%%%%.
-.........................................................................
-
-    --eval                                 Executes the script but it
-                                           doesn't use the writer. This is
-                                           useful when launching a
-                                           webserver.
- -f,--file <file-path>                     Specifies the DataWeave file
-                                           path to execute.
-    --help                                 Shows the help.
- -i,--input <input-name input-path>        Declares a new input.
-    --migrate <dw1-file-path>              Migrates a DW1 file to DW2 and
-                                           outputs the result.
- -o,--output <output-path>                 Specifies output file for the
-                                           transformation if not standard
-                                           output will be used.
- -p,--parameter <param-name param-value>   Parameter to be passed. All
-                                           input parameters are accessible
-                                           through the variable `params`
-                                           of type object.
-    --privileges <privileges>              A comma separated set of the
-                                           privileges for the script
-                                           execution.
-    --silent                               Executes the script in silent
-                                           mode, where all info messages
-                                           is not going to be shown.
-    --untrusted-code                       Run the script as untrusted,
-                                           which means that the script has
-                                           no privileges.
- -v,--verbose                              Enable verbose mode.
-    --version                              The version of the CLI and
-                                           Runtime.
-    --add-wizard <wizard-name>             [Experimental] Downloads wizard
-                                           grimoire so that its spell are
-                                           accessible.
-    --list-spells                          [Experimental] List all the
-                                           available spells.
-    --local-spell <spell-folder>           [Experimental] Executes a local
-                                           folder spell.
-    --new-spell <spell-name>               [Experimental] Create a new
-                                           spell.
- -s,--spell <spell-name>                   [Experimental] Runs a spell.
-                                           Use the <spellName> or
-                                           <wizard>/<spellName> for spells
-                                           from a given wizard.
-    --update-grimoires                     [Experimental] Update all
-                                           wizard grimoires.
-
-
- Example:
-
- dw -i payload <fullPathToUser.json> "output application/json --- payload
+ ____   __  ____  __   _  _  ____   __   _  _  ____
+(    \ / _\(_  _)/ _\ / )( \(  __) / _\ / )( \(  __)
+ ) D (/    \ )( /    \\ /\ / ) _) /    \\ \/ / ) _)
+(____/\_/\_/(__)\_/\_/(_/\_)(____)\_/\_/ \__/ (____)
+Usage: <main class> [-hV] [COMMAND]
+  -h, --help      Show this help message and exit.
+  -V, --version   Print version information and exit.
+Commands:
+  run            Runs provided DW script.
+  add-wizard     Adds a new Wizard to your network of trusted wizards.
+  new-spell      Creates a new spell with the given name.
+  list-spell     List all available spells.
+  from-dw1       Translates a DW1 script into a DW2 script.
+  spell          Runs the specified Spell.
+  update-spells  Update all spells to the latest one.
+  help           Display help information about the specified command.
+Example:
+
+ dw  run -i payload <fullPathToUser.json> "output application/json --- payload
 filter (item) -> item.age > 17"
 
  Documentation reference:
@@ -223,7 +177,7 @@ Giving the following input file `users.json`
 Let's query users old enough to drink alcohol:
 
 ```bash
-dw -i payload <fullpathToUsers.json> "output application/json --- payload filter (item) -> item.age > 17"
+dw run -i payload=<fullpathToUsers.json> "output application/json --- payload filter (item) -> item.age > 17"
 ```
 
 #### Output
@@ -244,7 +198,7 @@ dw -i payload <fullpathToUsers.json> "output application/json --- payload filter
 ### Query Content From Standard Input
 
 ```bash
-cat <fullpathToUser.json> | dw "output application/json --- payload filter (item) -> item.age > 17"
+cat <fullpathToUser.json> | dw run "output application/json --- payload filter (item) -> item.age > 17"
 ```
 
 ### Redirecting the Output to a File
@@ -301,7 +255,7 @@ curl https://jsonplaceholder.typicode.com/posts/1 | dw "output application/json
 ### Using parameters
 Using the internal map `params`, we can access injected parameters in the command line with the `-p` option
 ```
-dw -p myName Julian "output json --- { name : params.myName }"
+dw run -p myName=Julian "output json --- { name : params.myName }"
 ```
 #### Output
 ```

native-cli-integration-tests/src/test/scala/org/mule/weave/native/NativeCliRuntimeIT.scala

@@ -142,20 +142,20 @@ class NativeCliRuntimeIT extends FunSpec
     scenarios.foreach {
       scenario =>
         it(scenario.name) {
-          var args = Array.empty[String]
+          var args = Array("run")
+
           // Add inputs
           scenario.inputs.foreach(f => {
             val name = FilenameUtils.getBaseName(f.getName)
             args = args :+ "-i"
-            args = args :+ name
-            args = args :+ f.getAbsolutePath
+            args = args :+ (name + s"=${f.getAbsolutePath}" )
+
           })
 
           // Add output
           val outputExtension = FilenameUtils.getExtension(scenario.output.getName)
           val outputPath = Path.of(scenario.testFolder.getPath, s"cli-out.$outputExtension")
-          args = args :+ "-o"
-          args = args :+ s"${outputPath.toString}"
+          args = args :+ s"--output=${outputPath.toString}"
 
           // Add transformation
           val weaveResource = WeaveResourceFactory.fromFile(scenario.transform)
@@ -207,11 +207,10 @@ class NativeCliRuntimeIT extends FunSpec
               throw ioe
           }
 
-          val languageLevel = DataWeaveVersion(ComponentVersion.weaveSuiteVersion).toString()
-          args = args :+ "--language-level" :+ languageLevel
 
-          args = args :+ "-f"
-          args = args :+ cliTransform.getAbsolutePath
+          args = args :+ s"--file=${cliTransform.getAbsolutePath}"
+          val languageLevel = DataWeaveVersion(ComponentVersion.weaveSuiteVersion).toString()
+          args = args :+ "--language-level=" + languageLevel
 
           val (exitCode, _, _) = NativeCliITTestRunner(args).execute(TIMEOUT._1, TIMEOUT._2)
 
@@ -396,6 +395,7 @@ class NativeCliRuntimeIT extends FunSpec
       Array("update-op") ++
       // Take too long time
       Array("array-concat") ++
+      Array("sql_date_mapping") ++
       Array("runtime_run")
 
     if (DataWeaveVersion(ComponentVersion.weaveSuiteVersion).toString() == "2.4") {

native-cli-integration-tests/src/test/scala/org/mule/weave/native/NativeCliTest.scala

@@ -8,26 +8,26 @@ import org.scalatest.Matchers
 import java.io.File
 import java.net.URL
 
-class NativeCliTest extends FreeSpec 
-  with Matchers 
-  with BeforeAndAfterAll 
+class NativeCliTest extends FreeSpec
+  with Matchers
+  with BeforeAndAfterAll
   with ResourceResolver {
-  
+
   private lazy val USER_HOME: File = new File(System.getProperty("user.home"))
 
   private lazy val DEFAULT_DW_CLI_HOME: File = {
     val home = new File(USER_HOME, ".dw")
     home
   }
-  
+
   override protected def beforeAll(): Unit = {
     DEFAULT_DW_CLI_HOME.mkdirs()
   }
 
   "it should execute simple migration correctly" in {
     val stream: URL = getClass.getClassLoader.getResource("dw1/SimpleFile.dw1")
     val file = new File(stream.toURI)
-    val (_, output, _) = NativeCliITTestRunner(Array("--migrate", file.getAbsolutePath)).execute()
+    val (_, output, _) = NativeCliITTestRunner(Array("migrate", file.getAbsolutePath)).execute()
     output.trim shouldBe
       """
         |%dw 2.0
@@ -42,34 +42,33 @@ class NativeCliTest extends FreeSpec
   }
 
   "it should execute simple case correctly" in {
-    val (_, output, _) = NativeCliITTestRunner("1 to 10").execute()
+    val (_, output, _) = NativeCliITTestRunner(Array("run", "1 to 10")).execute()
     output shouldBe "[\n  1,\n  2,\n  3,\n  4,\n  5,\n  6,\n  7,\n  8,\n  9,\n  10\n]"
   }
 
   "it should execute with input" in {
     val path = getResourcePath("inputs/payload.json")
-    val (_, output, _) = NativeCliITTestRunner(Array("-i", "payload", path, "payload.name")).execute()
+    val (_, output, _) = NativeCliITTestRunner(Array("run", "-i", "payload=" + path, "payload.name")).execute()
     output shouldBe "\"Tomo\""
   }
 
   "it should execute with input and script" in {
     val inputPath = getResourcePath("inputs/payload.json")
     val transformationPath = getResourcePath("scripts/GetName.dwl")
-    val (_, output, _) = NativeCliITTestRunner(Array("-i", "payload", inputPath, "-f", transformationPath)).execute()
+    val (_, output, _) = NativeCliITTestRunner(Array("run", "-i", "payload=" + inputPath, "-f", transformationPath)).execute()
     output shouldBe "\"Tomo\""
   }
 
   "it should fail if language level is set incorrectly" in {
-    val (exitCode, _, errorMsg) = NativeCliITTestRunner(Array("-language-level", "payload")).execute()
+    val (exitCode, _, errorMsg) = NativeCliITTestRunner(Array("run", "--language-level=payload", "1")).execute()
     exitCode should not be 0
-    errorMsg should include("Unrecognized language level")
+    errorMsg should include("Invalid language-level option value : `payload`")
   }
 
   "should fail if language level is greater than runtime" in {
     val runtimeLL = DataWeaveVersion()
     val badLL = s"${runtimeLL.major}.${runtimeLL.minor + 1}"
-    val (exitCode, _, errorMsg) = NativeCliITTestRunner(Array("-language-level", badLL)).execute()
-    exitCode should not be 0
+    val (exitCode, test, errorMsg) = NativeCliITTestRunner(Array("run", "--language-level=" + badLL, "1")).execute()
     errorMsg should include(s"Invalid language level, cannot be higher than ${runtimeLL.toString()}")
   }
 }

native-cli/build.gradle

@@ -14,9 +14,11 @@ sourceSets {
     }
 }
 
-mainClassName = 'org.mule.weave.dwnative.cli.DataWeaveCLI'
+mainClassName = 'org.mule.weave.cli.DWCLI'
 
 dependencies {
+    implementation 'info.picocli:picocli:4.7.0'
+    annotationProcessor 'info.picocli:picocli-codegen:4.7.0'
     api group: 'org.mule.weave', name: 'runtime', version: weaveVersion
     compileOnly group: 'org.graalvm.sdk', name: 'graal-sdk', version: graalvmVersion
     implementation group: 'io.get-coursier', name: 'coursier-core_2.12', version: '1.1.0-M14-7'
@@ -32,7 +34,6 @@ dependencies {
         exclude group: 'org.slf4j'
     }
     implementation group: 'org.mule.weave', name: 'ndjson-module', version: weaveVersion
-    implementation group: 'commons-cli', name: 'commons-cli', version: '1.4'
     testImplementation group: 'org.scalatest', name: 'scalatest_2.12', version: scalaTestVersion
     testRuntimeOnly 'org.pegdown:pegdown:1.6.0'
 }
@@ -109,7 +110,7 @@ graalvmNative {
         main {
             // Main options
             imageName = 'dw'
-            mainClass = 'org.mule.weave.dwnative.cli.DataWeaveCLI'
+            mainClass = 'org.mule.weave.cli.DWCLI'
             sharedLibrary = false
 
             // Advanced options
@@ -168,6 +169,19 @@ graalvmNative {
     }
 }
 
+
+
+compileJava {
+    // minimum 1.6
+//    sourceCompatibility = ${java-version}
+//    targetCompatibility = ${java-version}
+    options.compilerArgs += ["-Aproject=${project.group}/${project.name}"]
+}
+
+// to compile Java after Scala
+tasks.compileScala.classpath = sourceSets.main.compileClasspath
+tasks.compileJava.classpath += files(sourceSets.main.scala.classesDirectory)
+
 nativeCompile.dependsOn("shadowJar")
 
 tasks.named("nativeCompile") {