Basic Input/Output ⌨🖥️

[lestrozi]

Based on previous discussions (#148 and #152), I think it makes sense to start this one.

Here's a proposal to support Basic (but not so basic as currently) Input/Output (backwards compatible):

proposal 1
flip dw+2 (first time) - activates extended mode

this turns 4dw, 5dw, 6dw and 7dw into "registers" (let's say: a, b, c, d)
then we can set values on those registers and flipping dw+n (n>2) is like calling "int n", which will perform an IO operation

for example:
a=0, b=65, c=?, d=? + flip [dw+3] -> prints b ('A')
a=1, b=? , c=?, d=? + flip [dw+3] -> sets b as 0x80 if there's input waiting or 0x00 otherwise (the current input reading could be used for reading bits)

If this works, I suppose we could start thinking about the graphical IO calls later

proposal 2
very similar to 1, but the "trigger" that causes the interrupt could be jumping to [4dw] (so you need to set the return addr in 4dw+w first) instead of just flipping dw+n
this would allow 4 different interrupts (you could also jump to 5dw, 6dw and 7dw)

EDIT: scratch that, it doesn't work if the interrupt returns a value into a register since you'd need to jump to it to read the value and you don't want to run the interruption again

---

Curious about your thoughts on that.

[tomhea]

Hold on, It's going to be a long one 😺

Cool work and ideas!

After reading what you posted, I did some research last night to see what "syscalls" we would like to support. This comment is all about the different syscalls fj should support, and how.

Are you familiar with linux's "Everything is a file" concept?
Shortly, it means that communication, devices, and syscall can be handled (and input/output data) using particular "files". for example, you get random numbers when reading from the file /dev/random.

I came to the conclusion that the current input-output mechanism could go very well with these kinds of files.

I got to the next list. I find it important to keep in minimal as possible, and not support some OS concepts that can be implemented on flipjump itself.

Flip jump IO interface

 0 -> 2w-1:    first fj op (usually jumps to ;8w)
2w -> 4w-1:    IO (as before)
4w -> 8w-1:    current file (4-lsb bits at 4w->6w-1, and 4-msb bits at 6w->8w-1. it's a hex.vec 2, usually initiated with 0)

The files

0. standard input-output - unchanged: keyboard input, screen output
1. standard error output - stop on input, write error output
2. standard random - stop on input, write random bits on output
3. syscall file - currently stops on input-output, but allows supporting more input-output on specific machines supporting it
4. get op count - stop on output, get by 64bit input the number of fj ops executed since the program started, not including this one.
5. FillRectangle(x, y, w, h, color) - output 4*64 + 4 bits (the size of the arguments), then the screen will set the rectangle colors. stops on input.
6. GetPixel(x, y) - output 2*64 bits (the size of the arguments) then an input of 4 bits will be available to read (stops on input when read is not available)

So we'll get key presses with file 0, and color the screen with files 5/6.
We can also call these modes IO-modes, but I'll use the "file" naming in this comment.

About the random - we can get the same functionality ourselves by getting only a 64bit seed, and calculating in fj the next random numbers. Yet, it does look simpler to just get one bit at a time, even though it might be a bit stronger.

About FillRectangle - It kinda bothers me that it's not its simplest form (SetPixel(x, y, color)), yet I see the advantages of using it once to color an entire rectangle, so I think its a good choice going with it.

I have thoughts about adding 2 more "Files" for setting pixels (which will be faster than the above-suggested options in some cases).

• WriteRectanglePixelStream(x, y, w, h) - output 464 pixels, then 4w*h pixels to color each pixel differently in the rectangle (top to bottom, left to right). stops on input.
• ReadRectanglePixelStream(x, y, w, h) - output 464 pixels, then read wh 4-bits pixel colors (top to bottom, left to right)

More Files

Also, I thought about adding the next file-interfaces (as these statistics are easy to implement), but I'm not sure if they are necessary:

• get flip count - stop on output, get by 64bit input the number of fj ops that flipped addresses >=2w, since the program started, not including this one.
• get jump count - stop on output, get by 64bit input the number of fj ops that jumped to addresses other than the next ones, since the program started, not including this one.
• get time - get a 64-bit current time in microseconds.

Actual Files

At first, I thought that it might be cool to add support for file system operation (as open/close/create/delete files); But this kind of abstraction just magically gives us all that we want.
The power of such a system will come from the file system itself, not from flipjump (i.e. the option to save data in a key: value way - "file_name": "data"). This thing, and any os-like options, can be programmed in flipjump at its current state, in its own memory. flipjump should only be the bare minimum.
Another syscall I thought of supporting is the one returning the path to the .fjm currently being run, but I guess it's redundant without the other file system operation.

Sockets

Another thing I think that might be supported is sockets, as communication can't be done with flipjump on its own (it can be treated as a new device, like the screen):

• CreateIPV4Socket(bind: 1bit, protocol: 8bit, IP: 32bit, port: 16bit) - output 57bits to create a socket (protocol 6 = TCP, 17 = UDP. maybe 1bit is enough if we would only like to support UDP/TCP and no other protocols in the future). creates an IPV4 socket, with 32bit IP and 16bit port arguments, using the TCP/UDP protocol. that will return an 8-bit number, waiting to be inputted, which is the socket file descriptor. stops on more input.
• Close - output an 8-bit file descriptor number, to close the file/socket behind it. stops on input.

Reading and writing to sockets:
if not binded (client socket):

• Writing to an open socket will write to the next packet that's going to be sent.
• Reading from an open socket will read (in a blocking way) the next bits from the received data from the socket.
  If binded (server socket):
• Stops on writing.
• Reading (blocking) - wait for the next connection, and returns the IP: 32bit, port: 16bit, and an 8bit file descriptor (for the socket to talk with this client).

Reading/writing to a closed / not-defined-yet file descriptor will stop the program.

Of course, all this sockets talk is cool but probably won't be in the near future, but I'd love to hear what you think about this subject.
The minimal way of doing this is just allowing to add a network device as a file, and doing all the communication with it via IO, so the syscalls/files I defined above won't be needed.

EOF

With using files this way, an end-of-file check should be supported.
It will help us check if the user finished inputting, or if new data is waiting to be read.
I thought of implementing it by flipping the 9'th bit (the 8'th index) of the current file (so that 5 bits will be used in 6w->8w-1) to look at a new kind of file - Theis an EOF file for each file.

• Reading from this EOF file will give us 0 if there is more input in the original file, and 1 if there is not (EOF). maybe return 2 if the file is closed (I'm not sure if it can be checked without trying to read. I think that it might be better to define a place to jump if the connection is closed).
• Writing to this EOF file means to flush the original file's output (send the currently written packet; or flush output to the screen)

---

I'd be happy to hear your thoughts.

[lestrozi]

Oh, that's interesting! I like how you're using the concept of "everything is a file" here, but perhaps based on how I view FJ - more like a single-op CPU rather than a programming language - I think it should be thought in terms of "serial ports" rather than "files" (for one thing, in order to actually talk about files you need to define a filesystem). But using this concept here is very clever and I think it solves all the problems indeed.

At first this could sound like a terminology change, but there's more to it.

Before I try to explain my thoughts, those are somethings that came to my mind after reading your post:

1. Having access to a /dev/random "file" doesn't sound right. I think the CPU could expose values like getTime (as you mentioned) and/or getTicks - and those could be used by libs to derive a PRNG.
2. "get op count", "FillRectangle", "GetPixel".. those don't sound like files. And we'd possible also need another "file" to set the screen mode/resolution.
3. The way you describe the files is that they basically implement fread() [no return] and fwrite() [no return]; and you also proposed feof(). I think it would be more clear to have fread() returning how many bits were read (0 or 1) rather than trying to emulate this behavior using feof(). Also, if everything is a file (not a stream!), we need fseek(). And perhaps fopen() or some way to set flags on how we want to interact with the file (e.g. blocking/non-blocking).

Ok, going back to my "serial ports" rather than files, I propose having 8 serial ports.

1. graphics card
2. CPU (I need to think more about that, but that would be way for a program to communicate with the CPU - and ask the time, getTick, getFlipCount, etc.)
3. keyboard
4. mouse
5. storage
6. NIC
   7-8) [unused]

What are those devices exactly? In the current implementation, they'd be python modules that implement (and here begin the similarities with your proposal) the following methods:

1. hasBit()
2. readBit() - blocking
3. canWriteBit()
4. writeBit() - blocking

(I didn't have much time to think how those methods would be called from FJ, but certainly from specific memory-mapped addresses - similar to what you proposed for files)

Then when you run flipjump, you could pass which modules you'd plug to which ports (those above could be the default).

What if we need more devices? Nothing stops you from writing a "hub" module that can address N number of devices. Reading is a nightmare though, since you first need to find out which device is sending the message, then map the message to this device handler - that's the only reason I'm not proposing having a single serial port and abstracting all that.

Before I think this in more details, how's that sounding so far?

[lestrozi]

I thought a bit more and would change a couple of things on what I suggested:

Running FJ with default arguments would have the current behavior, no "serial ports". This makes it easier for newbies to experiment with it and get something done.

Also, no limit on number of ports. When calling FJ you can specify which python modules will act as serial ports and, starting on 4dw, they'd get 3dw each (0dw=output, 1dw=input, 2dw=hasBit/canWriteBit flags).

[lestrozi]

Sample module: https://paste.ofcode.org/RPh5J7G5FzFsCQuyFzVbTw

[lestrozi]

Ok, one more comment
I think I realize now you wanted to keep an interface similar to the existing one so it would be easier to make the existing libraries writing to multiple "serial-ports" without having to know to what they're writing to.

That makes sense, so I think using what you said is fine, with a small change:

2w:    output (as before)
3w:    input (as before)
4w:    current device/serial-port (4-5 bits should be enough)
5w:    would store the hasBit and canWriteBit flags

2w+0; // getDevice(4w).writeBit(0)
2w+1; // getDevice(4w).writeBit(1)
;3w // (probably done using wflip) -> (jump to (3w ^ 0x80*getDevice(4w).readBit())) and zeroes 3w
4w+n; // updates the current device we're reading/writing
;5w // (probably done using wflip) -> (jump to (5w ^ 0x80*(getDevice(4w).hasBit() + 2*getDevice(4w).canWriteBit()))) and zeroes 5w

Not sure how clear I made it, but lemme know if you have questions.

[lestrozi]

I put together a proof-of-concept here: main...lestrozi:poc_pong

$ python3 src/fj.py programs/pong.fj

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. # . . . . . . . . . . . . . . . . . . . . . . . . . . . # . 
. # . . . . . . . . . . . . . . . . . . . . . . . . . . . # . 
. # . . . . . . . . . . . . . . . . . . . . . . . . . . . # . 
. # . . . . . . . . . . . . . . # . . . . . . . . . . . . # . 
. # . . . . . . . . . . . . . . . . . . . . . . . . . . . # . 
. # . . . . . . . . . . . . . . . . . . . . . . . . . . . # . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 
                                finished by looping after 0.001s (90 ops executed, 97.78% flips)

[tomhea]

I think I realize now you wanted to keep an interface similar to the existing one so it would be easier to make the existing libraries writing to multiple "serial-ports" without having to know to what they're writing to.

Exactly.

I think that we should use the terminology of "Devices" from now on.

I liked your suggestion, and I have a few changes.
As I see it, we need to support 3 actions

• check if EOF (non-blocking)
• check if there is available input (non-blocking)
• Flush the output (flush the output text / flush the screen / send the packet)

about the next, I'm not really sure why it is needed:

• check if can write (non-blocking) (why wouldn't it be? is I really needed?)

The 2/3 checks can be done as you suggested (jumping to 4w and reading from 5w).

I think that the best way to put the Flush bit is in 2w+2:

• It relates to the output, and 2w+0, 2w+1 both are about the output
• we can't really find a good place to put this bit organically in 4w+?
• It will still be backward compatible

2w:    output (as before) + Flush using 2w+2
3w:    input (as before)
4w:    current device number (not sure if we should use 4/8/more bits of devices yet)
5w:    would store the availableRead, availableWrite, and EOF flags

2w+0; // getDevice(4w).writeBit(0)
2w+1; // getDevice(4w).writeBit(1)
2w+2; // getDevice(4w).flush()
3w // (probably done using wflip) -> (jump to (3w ^ 0x80*getDevice(4w).readBit()))
4w+n; // updates the current device we're reading/writing
5w // (probably done using wflip) -> (jump to (5w ^ 0x80*(getDevice(4w).availableRead() + 2*getDevice(4w).availableWrite() + 4*getDevice(4w).EOF())))

About zeroing 3w and 5w - It is a change that the stl and many tests won't work without refactoring them. I don't think the refactoring should take much time, but we need to know that and update it so that pytest --all will pass successfully.
I know it seems like a logical change, yet it has some downsides. The reason I chose not to do it in the first place was to keep fj as simple as possible. With your suggestion, the Flipjump CPU will flip those bits by itself (to zero the word) - and that's something that the fj language can't really do - simply zero bits in memory. It's something it can't do. And I don't really like the idea that in 3w/5w it will happen, just as it is. I still want to keep it minimal as possible.
That's why when outputting bits I did flip the output bits (2w+0,2w+1) in memory. The reason I wasn't worried about flipping the 2w+0,2w+1 bits, is that they only address bits in 0→2w-1, so they won't be accessed anyway in the future.

Yet, it will be easier for newcomers, It's kinda more elegant, and it might save a few flipjump-opcode here and there.
So my real suggestion (not with all my heart yet) is:

• zeroing 3w/5w after jumping to them.
• while flipping the 0/1/2 bits of 2w, and the device-control bits of 4w, they won't really be changed (will stay what they were, which should be zero).
  What do you think about all that?

Btw I just saw your pull request, nice job!
I have a few notes on it, but first I created a new branch, deviced-io (main→dev→deviced-io), and we will make all the changes directly on it. So first change your pull request to be merged into it, and then update it to work with the new IO-communication-way we'll choose.
By the way, to run the important tests, I'd suggest running the tests with pytest --slow --medium --fast (and -n auto for speed) just to be sure before any pull request. There are also the "--hexlib" tests, but they should be run only after hexlib stl change. I know I'll do it.
Just so you'll know, you don't need to specify the tests/ directory, pytest will get it from its name.

[lestrozi]

In order to help making decisions such as the interrupt ones, I think we could start thinking which devices/methods we'd like to have.

Here's a proposal:

Graphics Card (16 bits variables (except line/row can be 8?))

• setScreenMode(n)
• getPixel(x, y)
• fillRect(x, y, w, h, color)
• setCursor(line, row)
• getCursor(line, row)
• beginText() # after this command, it would start printing chars like it currently does with dw+0 and dw+1; it would stop treating dw+0 and dw+1 as char-bits when a signal() is sent (what you're calling flush()), then it reverts to "waiting op" mode; btw, it can start on this mode for backward compatibility with BasicIO (but it won't work as a keyboard - in order to read from stdin you'd need to select Keyboard device)

Keyboard (8 bits)

no "op", it would just act as the current BasicIO device "read" works (in this sense, it doesn't really emulate a keyboard as it's not sending scancodes, but oh well..)
it could accept write() in order to set something on the keyboard (e.g. num/caps/scroll lock), but I wouldn't do it now

Storage

• open(filename, readOnly bool)
• close(filename) # probably easier than having to read a file descriptor from open and use it here to close
• beginReading(n) # if n == 0, reads until the file is over; stops reading if signal() is sent
• beginWriting(append bool) # start writing until signal()

NIC (TCP only - perhaps other protocols in the future?)

• connect(host, port) # host can be hostname, IPv4, IPv6.. I would abstract all
• beginReading(n)
• beginWriting(n)
• // bind(port)? it could have an auto-accept single-connection bind that would send the IP that connected when a connection is established, then beginReading/beginWriting() must be called - for the future?

[tomhea]

Hi, view the new comment I wrote, I took some of the ideas we had into it.

[lestrozi]

Where?

[tomhea]

Just uploaded it down here, but couldn't finish it, so I'll fill the missing spots in a few hours :)

[tomhea]

Just finished updating. I was busy in the last week more then usual, but it's done 😌.
I know it's a bit different from the 4w-6w device-control we talked about, but I think that this model also has its own elegance, and it seemed simple enough to me :)

I'd be happy to hear your thoughts, but I'm kinda closed on it.

We should implement the changes with two tests - one that

Three points I would like your opinion about are:

• Do you think we really need a TextScreen device? We can implement all of its functionality on a graphicScreen, with the place_ascii_character macro I'll create from the unicode font bitmap. It must also be implemented that way in cmd too.

def move_cursor(y, x):
    print("\033[%d;%dH" % (y, x))

• Do we even need to read a pixel from the screen? It is not a feature that is supported / usually used on the cmd for example (for reading a letter). Also, games and such usually maintain screen-objects data structures and would be able to tell the pixel's value without querying the screen. I think we can get rid of this function.
  If we choose it is needed so, It seems that the cursor can move with:
• do we need the random-bit and fj-ops-count queries?
  The usage of the fj-ops-count seems esoteric.
  We can get "random bits" from the time query, but it can be "manipulated", and It's it only depends on the number of fj ops executed since the last call and the result of the last call (and input). So programs without inputs would probably only succeed in generating about 16-20 random bits of time. The seconds, minutes and such can be predictable, and consecutive calls can be predictable too.
  16-20 bits are enough for the randomization of a game (using these bits as the starting seed of a RNG), but for encryption key generation? It won't be. That's way I wanted the random bit query.
  On the other side, a bigger seed can be given in the Standard Input for example, for the programs that will need it. What are your thought about it?

[tomhea]

The new IO model

Main concepts:

• open device manager context using IO+2, and thus opening, switching, and closing devices.
• supports querying available input/output/EOF
• supports getting system info.
• completely backward-compatible. the .fjm format doesn't need to be changed, so no new version is needed.
• future-compatible - left reserved places for future additions in protocols.

It doesn't support a general flush command as discussed above, nor does it support interrupts.

The IO model structure

Memory Layout:

Addresses [0w, 2w): starting address
Addresses [2w, 4w): used for IO
Addresses [4w,  +): regular memory

Some definitions:

IO = 2w
dbit = w + #w

General IO functionality:

Flipping IO+0: outputs 0
Flipping IO+1: outputs 1
Flipping IO+2: start device manager mode

Jumping to IO: fills the IO+dbit bit with the next input bit (and then jumps accordingly).
Some devices can put more bits (e.g. the 4 bits starting from IO+dbit onwards). If not specified, a device outputs 1 bit at a time (so input from it is 1 bit at a time).

Both input and output might block until the device is ready (to yield an input or to get your output).

The Device Manager

This mode is used for connecting to new devices, switching between devices, and querying data about the devices.
When in this mode, the next options are possible:

0 (Query data):
 00 - available-input (is there non-blocking input ready to be inputted? will be available as a 1-bit input. 1 is available).
 01 - available-output (is the device ready for my output? will be available as a 1-bit input. 1 is available).
 10 (system data):
   00 - time (in 100ns since 1970?) (will be available as 64-bit input)
   01 - number of fj ops executed from the start of the program, not including this one (will be available as 64-bit input)
   10 - get a random bit (will be available as a 1-bit input)
   11... - reserved
 11... - reserved

1 (Device manager)
 000 - open device (followed by 8 bits for device type, and optionally more bits for the device configuration). This opens a new device as specified, and implicitly switches to it. The opened device number will be available as an 8-bit input. 0xff means failure.
 001 - switch device (followed by 8 bits for the device number) - this will redirect any input/output from/to the given device.
 010 - get current device - get the 8bits device number, and then the 8bits of the device type (0x00 for default devices).
 011 - close this device (if still connected, implicitly calls "flush". implicitly switches to 0xff). It can't be called on device numbers 0x00/0xff.
 1... - reserved

At the start of each program, the current device will be 0x00, the standard IO.

The Devices List:

Device types are numbers from [0x01, 0xfe].

Standard IO [defaultly opened as device number 0x00. can't be opened or closed]:
The standard Input and Output. Good for tests (piping predefined inputs), and for piping FlipJump programs (connect one's output to another one's input).
This is the starting device, and it's available from the start. It's up to the running environment to decide from where the input will arrive, and where the output will go to.

Broken IO [defaultly opened as device number 0xff. can't be opened or closed]:
Fails and stops on any Input / Output.
It is good for testing parts of a program that shouldn't, and for catching errors.
This device can't be closed, so 0xff will always be the Broken IO device.
When closing a device, the current device will implicitly be changed to the Broken IO device.

Keyboard [device type 0x01]:
Input an 8bits representation of the entered key (blocking read).
Use the available-input query to see if there is a pressed key. If there is not, the read will block until a key is inserted.

TextScreen [device type 0x02]:
First waits for the program to output the screen's width (8bits), height (8bits), and text format (8bits) (0x00 - 8bit ascii characters, otherwise - reserved).
Then, communicates according to this protocol:

0xxxxxxxx - output the ASCII char x to where the cursor points, then advance the cursor.
1
 00 - get_cursor() - the cursor (row, column) will be available as an input.
 01 - set_cursor(row, column) - the cursor will be set to the given coordinates.
 10 - flush() - force the screen to update.
 11... - reserved.

Each one of row, column is 8bits.

GraphicScreen [device type 0x03]:
First waits for the program to output the screen's width (16bits), height (16bits), and pixel format (8bits). We can support 0x00 + 0x02 for starters.

0x00 - 1bit black-white
0x01 - 4bit grayscale
0x02 - 4bit RGBI
0x03 - 8bit grayscale
0x04 - 8bit RGB
0x05 - 16bit RGB
0x06 - 24bit RGB
0x07 - 32bits RGBA
otherwise - reserved

Then, communicates according to this protocol:

000 - read_pixel(row, column) - the pixel's color will be ready as an input.
001 - write_pixel(row, column, color) - the pixel's color will be updated.
010 - read_rectangle(row, column, height, width) - the pixels' colors will be ready as a consecutive input, top-to-bottom, left-to-right.
011 - write_rectangle(row, column, height, width) - the pixels' colors will be updated from the next consecutive outputs, top-to-bottom, left-to-right.
100 - fill_rectangle(row, column, height, width, color) - fill the entire rectangle with this color
101 - update_screen() - force the screen to update.
11... - reserved.

Each one of row, column, height, width is 16bits.
Rectangle definition: (row, column) is its top-left pixel, and its sides are height and width.

Optional - allow reading 4-bits at a time (hex-input) for a faster read.

Future Devices

The next devices don't need to be supported yet, but their concept is important.

Network Interface Card [device type 0x04]:
Allows creating a UDP, TCP, and a general IP Socket (connect(IP, port, type, bind?)).
For each socket, allows to read/write from it.
The reading / writing is done with the regular input output. Each message will start with a 16-bit integer - the message byte-size. An outgoing packet will be sent right after this number of bytes was outputted.
an input packet will stop generating input right after this amount of bytes where inputted.
If binded, reads the next connection (check ready-input bit to see in there is a next connection). Returns the IP, port, and device number for the new socket connected to it.

Hard Drive [device type 0x05]:
Actually, this is still an enigma for me.
I think that a hard drive should be memory-mapped (for example to addresses 0x80000000-0xFFFFFFFF).
I think that the other option, of IO-controlled random-access Hard-Drive, would just add in a not-so-legit way the primitive of random access to bits, and thus will improve the strength of our basic FlipJump.
I think that the device manager should allow memory-mapping of the latter part of the memory (addresses 0x80000000-0xFFFFFFFF) to the current device needs. And for this case, it will be mapped to the connected hard-drive memory.
The device could still be controlled (for example, flushing output) using the Input/Output at [2w, 4w).
What do you think about it?

Infinite Memory [device type 0x06]:
An "extension" to the current flipjump language.
Memory maps the last half of memory to the current memory-view. For an n-bit system, it is initialized with [2^(n-1), 2^n).
Outputting 1 will advance the whole view by 2^(n-1) bits (e.g. [2 * 2^(n-1), 3 * 2^(n-1)), then [3 * 2^(n-1), 4 * 2^(n-1)), ...).
Outputting 0 will move the whole view 2^(n-1) bits backwords. Fails and stops when outputting 0 on a [2^(n-1), 2^n) view.
Will be used in the future to implement an Universal Turing Machine, and to show that flip jump is indeed turning complete.

Note that in this scheme, every device can open a new device (for example, a binding socket can create a socket for a new connection). That's part of the devices-model power.

Throughout all this comment - if a program tries to execute a reserved op - simply fail and stop.

Implementing the screen

I did some basic research. It seems that a good way of implementing a screen is with Tkinter and Canvas.
25 seconds of youtube about it.
I read somewhere that it's slow if you want to set each pixel individually, and that using Tkinter with PhotoImage might be better.
Also, I think that it doesn't support getting the pixel values. So we might drop support to it, maintain a pixels array by ourselves, or find a better library / way of showing a screen.

Now, about printing Text on a graphical screen.
That's one of the things that seemed problematic at first and kinda stopped me from getting into implementing a screen before.
I found this awesome 8*8 unicode font bitmap. I have a good way in mind of making a fj macro out of it (place_ascii_character? maybe a new name), that gets an ASCII variable, and prints the 8*8 pixels representing it to the screen. And I'm excited to start working on it!
So keep in mind that printing text on a graphical screen won't be a problem.

[tomhea]

That way you can extend this infinitely (the cost is having to handle variable-length coding).

Nice, I took your changes

About the hard drive, I agree, and I leave it there as it is. let's not work on it right now and ad it only if we need it.

About turning completeness and flipjump - You are right, yet it still needs infinite memory to be proven real. so the extension is needed. Let's keep it as an idea for now, and implement it only when e get there, as the hard-drive.

I was thinking something like:

Ok I get your idea, it is more simple in the sense that it uses only the IO+0 and IO+1.
Yet, I think I'll proceed with the one I suggested above for the following reasons:

• it complicates things a bit: If we want to output a variable by its bits, we will have to check if it is 11111111, and if so we should print the 11111111 once again (or anything that we will choose). We should keep track of the number_of_outputted_bits % 8, and the current last bits outputted, to be sure that we know if we got there, as we won't always output in chunks of 8 bits.
• IO+2 creates a simple separation between the output model of IO+0 | IO+1, and the device control.

[tomhea]

Also, I searched how can you check EOF on files in python, and it turns out, you can't.
More particularly, You have to read some bytes, and if it's end_of_file - you will get an empty bytes-sequence.
so to find out if it's EOF you first have to read. It cannot be checked without reading (in files it might be possible... TODO).

The two devices that need the support of the EOF functionality are StandardIO and Sockets:

• The sockets can actually get away with it. I think that we should standardize that we send the message size as the first 16bits (our simplified "header"). So reading from a socket won't need an EOF, as we will know how much to read.

The standard IO is tricky. Any protocol can be used here.. I propose to choose one of the next approaches:

• As the program knows how to handle the Standard IO and what "protocol" is used on the other side of its input/output, we don't need to add an EOF functionality. It should be implemented in the protocol itself.
• We need to add an EOF functionality, but we remember that we can't know if it's EOF until we do the read. so we will flip the bit after the now we flip to indicate a 1-input. so a 2-input will be EOF.

I believe that the first option is much better, will be much simpler to implement, use, and explain, and much more flipjump-ic.

Also, do you think that we still need the available-input query?

If you notice, I came to mind that it's great to keep the IO simple and leave the more complicated stuff to the protocol itself, as you suggested before, and not to try to generalize it and complicate the IO.

[tomhea]

Update: I pushed to deviced-io the FixedStandardIO and some more classes (inspired by your pull request).

Also, what do you think about the three questions I raised above?

[lestrozi]

If we want to output a variable by its bits, we will have to check if it is 11111111, and if so we should print the 11111111 once again

that's a good point
indeed that would be much harder, but... FJ interpreter already "process" IO as there's no way to output a single bit, so I don't think it would be a stretch to also have the interpreter dealing with the 0xFF case

that said, if you prefer to go on with your approach, I think we should try to make them as modular as possible so they can be reused by other esolangs. I'm planning on starting creating this "pipes device manager" (based on bytes, not bits). I played a bit with creating a TkInter window and I think it can become a "virtual machine" where you pipe everything to the process (including text) and it prints text in this window, capture keypress, switch to graphical model, etc.

It would be nice if we can reuse eachothers stuff, since the goal is the same

Also, I searched how can you check EOF on files in python, and it turns out, you can't.

TIL: EOF is not defined in POSIX
what feof() in C does is actually return if a previous read has tried to read past the end of the file, but a following read might succeed! (source: https://stackoverflow.com/questions/12337614/how-feof-works-in-c )

I suppose we could emulate this behavior.. but, that said, I always preferred not having eof :D

Also, do you think that we still need the available-input query?

hm, without it every time we need to read something without blocking the procotol needs to reply, e.g. 0 if there's nothing and 1+data if there is, right?
I think that's fine

[tomhea]

I think we should try to make them as modular as possible so they can be reused by other esolangs. I'm planning on starting creating this "pipes device manager"

What do you mean by that? You want to write one class that has 4 functions (output 0, output 1, start device manager, read bit) that will do all the magic by itself?
That sounds great.
I didn't understand your use of the word pipe there, maybe I'm missing a point?

Also I meant to ask about if we need the available-output, not the input.

I always preferred not having eof :D

I'm with you

It would be nice if we can reuse eachothers stuff, since the goal is the same

Yeah I agree. I didn't use your code till now since the model got changed in the way and we start thinking about a tkinter-screen. Let's work on eachother stuff in the deviced-io branch. I refactored the fjm-run module so it will be easier to change, so notice the new changes.

Just so we will be on the same page - we should commit for every small change (it's OK to have many commits in a pull request), and try to pull request one thing at a time.

Do you want to CR each other? We can try to give it a go for a few days and see how it is. It has it benefits.

[lestrozi]

I didn't understand your use of the word pipe there, maybe I'm missing a point?

so, my current proposal is basically to have a "vm.py" that can have devices plugged to it (it opens a TkInter window and can accept devices such as keyboard, "filesystem folder", ...)
the way to control this program is not calling by methods on it (like we did for e.g. writeBit, readBit) but rather to read from its (non-buffered) stdin/stdout
so any esolang that can "speak bytes" can control it using a standard unix-pipe to "vm.py" running as a standalone program

the biggest disadvantage for FJ is that it will only be able to control it using whole bytes (so the protocol can't be something that expects 3 bits, for example, as they'd need to get flushed).

Also I meant to ask about if we need the available-output, not the input.

yeah, it sucks to not have it because we can get blocked.. but oh well.. the protocol could have a way to signal that it's going to be busy

Yeah I agree. I didn't use your code till now since the model got changed

yeah, don't worry, it was just a prototype
I mean for the "pipe device manager" or however you prefer plugging it to FJ

Just so we will be on the same page - we should commit for every small change (it's OK to have many commits in a pull request), and try to pull request one thing at a time.

Do you want to CR each other? We can try to give it a go for a few days and see how it is. It has it benefits.

yeah, I think that's good

disclaimer: work is heavy lately and I'll probably only have 2-3 hours/week to sit down and write code (but I can spend a few minutes CR everyday) during next month
that said, I still want to finish the TkInter prototype this week (but feel free to move forward with your stuff, we can always merge what makes sense)

[tomhea]

I think that the pipe idea is cool, and I find it a bit overkill. I do prefer the out0, out1, device_manager, and input approach over the bytes one, as it allows any sort of communication. E.g. to allow pictures with 3-bit colors (it might not divide well by 8 so the program might need to keep track of the alignment of outputted bits), and to keep the protocols as generic as possible. The bytes approach can't handle a non-byte communication using the stdin/stdout (for example, piping two fj machines (output1->input2)).
I want to keep flipjump as a language as simple as possible.

You can still take it as a project and make a vm.py that works with any languages in a byte protocol. It could still be connected to the standard IO port of a flipjump program, and be used with it. I might even contribute to such a project. But let's first build the one that works with the suggested protocol above and make it work :) I updated it along with our discussion and conclusions, so it is up-to-date.

[lestrozi]

Cool! Just to show a very basic/hackish prototype of what I'm thinking: https://pastebin.com/1jq14qRv

$ echo "text" | stdbuf -i0 -o0 -e0 python3 a.py
will write "text" in the text mode

$ echo "textzAasdf" | stdbuf -i0 -o0 -e0 python3 a.py
will write "text" in the text mode, but "zA" makes it switch to graphical mode and "asdf" draws some rectangles (one for each letter)

$ echo "textzAasdfz@TEXT" | stdbuf -i0 -o0 -e0 python3 a.py
now "z@" switches back to text mode and print "TEXT", so you see "textTEXT" in the text mode

[lestrozi]

btw, a way to support bit output in this pipe scheme would be to output char 0 for bit 0 and char 1 for bit 1; then the "virtual machine" process them and, for example, print a char after 8 bits.
It's a waste due to the limitations of the architecture it's running on top of; but it doesn't interfere with FJ "cpu"

[tomhea]

Nice prototype!

I've gotta say I don't see the use-case of switching between Text and Screen, do you see how it helps?

The main reason I don't just adopt your byte-pipe idea is that I want to allow piping 2 fj programs one to another. I don't see how it is possible in this case. (fj-prog-1 outputs a bit -> fj-prog-2 inputs the bit). And to allow outputting a general output stream, e.g. a 0/1 bit for every number in input if it was prime.

I just can't limit fj programs that want to use a screen as output from getting a bit-stream input (and not a byte-stream).

Do you see my point? We should find a way to use It both

[lestrozi]

I've gotta say I don't see the use-case of switching between Text and Screen, do you see how it helps?

I don't have a strong opinion on it, but I think "text mode" having some features like a 80x25 screen, monospaced chars, auto line-wrap, auto-scroll, perhaps ANSI support; while graphic mode has a specific screen resolution, text is pixel(not char)-aligned, etc.

I don't see how it is possible in this case.

Not sure what's wrong with having an --io-mode flag that defaults to char, but can be set as --io-mode=bitchar
In --io-mode=char FJ emulator has the current behavior (ie, buffer bits until 8 is seem, then output a byte); in --io-mode=bitchar, FJ emulator "flushes" every bit by prefixing them with seven zero-bits.

This would allow you to run fj.py --io-mode=bitchar prg1.fj | fj.py --io-mode=bitchar prg2.fj (or more complex pipes if you want to pass their I/O to each other).

This is a limitation of the platform in which the emulator is running (as you currently already have a limitation, having to deal with I/O as chars).

The important part here, IMO, is that if you manage to create a real FJ CPU and somehow connect their I/Os, prg1.fj and prg2.fj would behave the same as in the emulator. How I/O is handled is completely abstracted from the program by the emulator/real CPU.

Do you see my point? We should find a way to use It both

Even if I don't convince you, I think we can still share a good deal of the devices, just having to change the protocol (which would be required to deal with bitchar anyway).

[lestrozi]

I'll try to make the first commit of a less hackish prototype today and ask for your opinion

[lestrozi]

There it is :)
https://github.com/lestrozi/pipevm

Not too different from the initial one, but more structured and using better patterns. If you check pipevm.py, you can see it allows for bit input (as bytes) on line 13. Currently it only supports byteAligned input, but this is not too difficult to change since I'm internally using a BitStream

[lestrozi]

I promise I won't post here every time I make a change on pipevm :) but now it has a PONG prototype (draw/move paddles/balls).
It's in python, but I'm thinking about doing a brainfuck version

[tomhea]

Cool!

I saw your repo, and I see your point. It is cool that it runs as a standalone and works with the terminal's piping.

Is it Linux specific?

By the way, I've worked on improving the assembler's code and speed in the last week, and I'm happy to say that it is significant! It is tested, and now available on the dev branch.

I will keep improving and documenting the current repo in the next week, but if you are up to it, you can start programming the pong game :)

[lestrozi]

Is it Linux specific?

for now, yes, but I think it can be extended to Windows using temporary files

improving the assembler's code and speed in the last week, and I'm happy to say that it is significant

cool!

if you are up to it, you can start programming the pong game :)

yeah, I've done it in Python already and I'm doing it in brainfuck now
since the BF community is pretty big, I'll see if I get some people interested first and perhaps improve the binary protocol; and then I can start working on the FJ bit version

by the way, I think I'll also try to implement a shell (well, at least with "ls" and "cat"; but it could also execute programs (./a.bf) if I manage to plug a BF interpreter (written in BF).. or later a FJ interpreter written in FJ :D

[lestrozi]

Nice! I'm making some good progress on the PipeVM in the brainfuck branch ( https://github.com/lestrozi/pipevm/tree/brainfuck ).
So far I have a working PONG and 90% of a simple shell (echo, ls, mkdir, rm, cat). It's basically missing echo with redirect (echo word >file).

I think I'll manage to post some demos early this week.

[lestrozi]

Prototype + demo (brainfuck) merged in https://github.com/lestrozi/pipevm/

You ended up with a device interface that only implements read_bit and write_bit, right? My interface is pretty similar: https://github.com/lestrozi/pipevm/blob/main/devices/__init__.py

We could try to make them compatible instead of duplicating the code, but I guess this depends mostly if you like PipeVM "design" (the protocol is very negotiable, as I think it makes sense to implement a bitwise version). In any case, even if you don't want to use a UNIX pipe (or something similar), you can probably reuse some of the devices and the VM implementation.

[tomhea]

That's cool! I loved both of your demos.

How did you captured your demo? I might do that too.

I do hold the belief that fj/bf should be below the os level, and therefore won't have access to a file system. But I'll take an idea when I'll get there. I think that a 3rd bit / escape byte is a good way of doing so, and I might get there soon.
You can take a look at the python src code in dev and see the change and the much cleaner code.

[lestrozi]

How did you captured your demo? I might do that too.

I have one Linux laptop with X11 and another with Wayland. In X11 it's much easier, pretty much anything will work (e.g. SimpleScreenRecorder). For Wayland I had to use OBS Studio.

I do hold the belief that fj/bf should be below the os level

Ha, we agree here. The idea of my Virtual Machine is that an OS would need to be implemented on top of that. But implementing an OS wouldn't be a cool "first project", so I cheated and offered PRNG, a filesystem implementation, etc. as VM devices (you can think about them as "smart devices" (e.g. a RaspberryPi) that was plugged to a dumb VM).
But yeah, ideally the devices should be dumber and only offer basic block-device operations and so on.
For simpler stuff like keyboard/mouse, we could even expose the raw device interface.
For a graphic card, that's not really possible.

If I get some free time I'll try to make my BF shell be able to execute another BF program. That would be a cool FJ project also, but it's probably really difficult (having to deal with relative memory positions and so on).

[tomhea]

I do have an implementation of pointers in stl/ptrlib.fj if that helps (of course, they can read addresses of flip jump bit-variables, not arbitrary bits. but I can see how a simple "exec" can work if it's not crucial to return back to the calling program).

Two tests are using ptrlib.fj:

• ptr.fj
• func{i}.fj

I also intend on creating a bf interpreter when I'll finish my current refactoring.

[tomhea]

Hi @lestrozi !
I just saw your brainfuck pong game using pipevm and it looks fantastic.

would you be open now to integrate it with flipjump? I am much more open to that then I was before 😄
It can be an IODevice named PipeVM.py.

A flip-jump pong would be awesome.
What do you say?

btw, I started thinking on a brainfuck -> flip-jump compiler, so when it'll be finished - we could take the brainfuck source you wrote and transform it into a flip-jump program.

[lestrozi]

Hey there! Sorry for the delay
I think the idea is cool, but I don't have much time at the moment. I suppose I can help on the VM implementation, but you'd probably write the FJ code 100x faster than me as you know all the libs :D

By the way, I believe it will be EXTREMELY slow. Not only you have to check collisions for every frame (which is "cpu bound"), but also send/receive a bunch of I/O instructions in order to move the paddle up/down.

That said, it could be a good project to use when optimizing cpu/io of an FJ compiler/interpreter.

[tomhea]

What do you mean by check for collisions?

[lestrozi]

basically the main loop
something like
{ move ball in the x/y direction it was going, check if it reached the vertical line of the paddle (if it did, check if it collided with the paddle and change its direction; move the computer's paddle towards the ball }

[tomhea]

Hi @lestrozi ! I'm open to that :) Sorry, I guess I forgot to comment back here and noticed it just now.

Also, a few updates:

• I'm currently working on making flip-jump a pip-installable package, so it could be used as a package in my brainfuck to flipjump compiler.
• The brainfuck to flipjump compiler works! see bf2fj. You can try it by running the bf2fj.py:main().
• I'm trying to make a flipjump read-the-docs site, for both the compiler, and the flipjump-standard-library, to make it more accessible :)