Odd behavior when programs prompt and then try to read from the repl

[kkinnear]

I have a command-line style Clojure application that prompts for input from the terminal when run. It works fine when I generate an uberjar with Leiningen, and it used to work fine in lein1 when I ran it at the repl. However, in lein2, when it prompts for input, the output at the repl includes an additional prompt

       #_=>

For example, if I create the function:

  (defn read-w-prompt [prompt] (print prompt) (.flush *out*) (read))

I can then use this function to read some input, like this:

  (read-w-prompt "Organization: ")

If I build an uberjar with this code in it and run it, when I want to enter "Accounting", it prompts like this:

Organization: Accounting  

which is what I would expect (and what I want), but from lein repl when I run (-main), it prompts like this:

Organization:      #_=> Accounting

This is certainly understandable, but makes working on the application difficult, in that I don't have the same user experience as someone actually running it in production.

Analysis:

This is because REPL-y is using jline2 to add an "empty-prompt" to lines where a read is generated by a Clojure program. This makes perfect sense if the Clojure program doesn't prompt for input (as might happen when experiementing at the repl), but doesn't work as well when the Clojure program already prompts for input (as mine does) because it is expecting console input.

The --prompt value in REPL-y and the :repl-options {:prompt (fn ...)} in Leiningen will allow me to change the prompt at the read in the repl, but that doesn't affect the empty-prompt that is
set by REPL-y in jline.clj when the Clojure program executes a read on its own.

Possible Solution:

It would be nice if there were a new command-line argument for REPL-y and :repl-options value for Leiningen. Perhaps --empty-prompt and :empty-prompt, respectively.

Additional Details:

I hacked a clone of REPL-y to add --empty-prompt, and found that it isn't quite that straightforward. It wasn't hard to get a user specifiable empty-prompt, but even when I did that, jline2 seems quite sure that it knows where it is on the line. Thus, the .setBlinkMatchingParen call, which is otherwise nice, causes any parens that you might enter to blink the wrong character.

I tried defining the empty prompt function to include access to the jline-reader, so that I could turn off the blinking paren capability in my user-defined empty-prompt. However, it turns out that the .setBlinkMatchingParen function in jline2 is defined in such a way that, while it takes a boolean in the call, only true has any effect. Calling it with false does nothing. So I figured out how to turn it off myself:

(defn no-blink [reader]
  (.bind (.getKeys reader) ")" Operation/SELF_INSERT)
  (.bind (.getKeys reader) "}" Operation/SELF_INSERT)
  (.bind (.getKeys reader) "]" Operation/SELF_INSERT))

So, now (if I execute this new function I created in jline.clj) the wrong characters don't blink. Which is nice.

But then, if I type in one character and hit delete, jline2 thinks that it is at the start of the line, and wipes out everything. Which is disconcerting, though not crippling.

I tried everything that I could think of to get jline2 to "know" where it was on the line. I tried sending the output through the jline2 reader instead of just to out, thinking that perhaps it would keep track of characters output without a \n, but no luck there. I contemplated watching the output in REPL-y and not sending the output to jline2 until a \n was received, in order to use any existing output as the "prompt" for jline2, so it would know that there was something already on the line.

This seemed like overkill for what seemed at first like a pretty simple problem.

Summary:

It would be nice if I could turn off the empty-prompt. I don't need to define a new one, but that might be the most general way to get the job done. It would be nice if I defined my own empty-prompt that I could turn off the blinking parens. The delete character behavior I would just live with, personally. I don't think the only code that I can imagine to fix that would be worth the additional complexity (and edge conditions that I would expect to occur).

[trptcolin]

Thanks for the detailed and clear issue report. Nice to meet you at the Conj.

This makes sense now, I think: jline2 manages the entire line where input is happening and will repaint the line on a variety of inputs, so it really expects to be completely in charge of what's printed. Keystrokes like Ctrl-L and the various motion bindings can result in the same sorts of problems you found with blinking parens.

The only good fix that comes to mind is to turn off jline interpretation entirely on lines where input is requested by evaluation (rather than the REPL loop).

That'll likely need to be pretty invasive, unfortunately, shifting from the with-jline-in macro wrapping the entire REPL session to a single input. But I think it's the only way to split these concerns up in a way that won't end up buggy-looking . It'll also help move towards #61 and #68, which I've known for awhile were issues, but this is a concrete reason to knock them out.

I'll dig into this soon.

caveat: I'm not sure what should happen on input like this:

(read-w-prompt "Organization: ") (+ 1 

, where a complete form requiring input is entered, and a second one is started but not completed. I'm thinking the non-jline read-w-prompt should come first, and then input from the (+ 1 form should resume, but the implementation of that hasn't crystallized in my head yet.

In the meantime, as a workaround, you can bypass jline entirely (and use rlwrap just as it worked in lein1) by doing something like this (will only work in a project context): lein trampoline run -m clojure.main/main. Or, to make things easier, you could set up an alias in ~/.lein/profiles.clj:

{:user {
        :aliases { "repl-no-jline"  ["trampoline" "run" "-m" "clojure.main/main"] } }
}

and run lein repl-no-jline in your project.

[andykitchen]

A huge bump to this one, it also breaks things like debug-repl badly :(

Aside from a few other things Ctrl-D is not sent through correctly
meaning that you can't actually exit the nested repl.

[trptcolin]

Oof, bad commit message, sorry. Haven't had time to address this one yet, but still thinking about how it'd work.

[trptcolin]

@kkinnear OK, I think I've finally got this solved on master - JLine is turned off for input reading, and your example appears to work well enough.

@andykitchen I also tried out debug-repl, and it mostly seems to work properly now. To exit, you need to do () - making Ctrl-D complete the input-getting operation would require some surgery in nREPL that'll take more thought.

I'd appreciate any testing anyone has a chance to do, since these changes are very invasive - using JLine in a very different way. lein trampoline run in a REPLy master checkout should do it. I'm on OSX, haven't yet had a chance to try it out on Windows or Linux, so any testing on those OSes would be especially appreciated.

[trptcolin]

OK, things seem to be working fine on Linux as well, so I'm going to go ahead and close this. 0.2.0 will include this fix.

[kkinnear]

Colin,

Hey, great!

Thanks for dealing with this. I'm looking forward to using it, it will make my
life a lot easier.

Thanks again,

Kim

On Apr 7, 2013, at 8:55 PM, Colin Jones wrote:

OK, things seem to be working fine on Linux as well, so I'm going to go ahead and close this. 0.2.0 will include this fix.

—
Reply to this email directly or view it on GitHub.