Illegal instruction when building on a p1010rdb board with a e500v2

[alepesoli]

I'm trying to compile and use v8 on a Freescale p1010rdb development board with a e500v2. I have successfully compiled it only with --without-snapshot option, and then if i try to execute it, i get an "illegal instruction" error.

If i try to compile it with the following command:
make ppc vfp3=off
(snapshot option enabled)
i have again an illegal instruction error, this time during the execution of "mksnapshot". Tracing this second case with gdb gives me the following output:

gdb) x/10i $pc
=> 0x4b628760: or r5,r4,r3
0x4b628764: clrlwi. r0,r5,31
0x4b628768: bne- 0x4b62877c
0x4b62876c: srawi r4,r4,1
0x4b628770: srawi r3,r3,1
0x4b628774: subf r3,r3,r4
0x4b628778: blr
0x4b62877c: mflr r0
0x4b628780: stwu r0,-4(r1)
0x4b628784: stwu r31,-4(r1)
(gdb)

[andrewlow]

The e500 series is supposed to support all of the instructions we use in the build - however you may be the 1st to actually try it.

Given that the crash doesn't look too suspicious (why would you crash doing a register OR instruction?) - my guess is that this is related to the instruction cache flushing sequence.

The machines we test against all have 128k sized caches.

There are two functions in the code that deal with the cache
macro-assembler-ppc.cc -> MacroAssembler::FlushICache
cpu-ppc.cc -> CPU::FlushICache

The constant is in constants-ppc.h

// For FlushICache
// This constant will be different for other versions of PowerPC
// It must be a power of 2
const unsigned int kCacheLineSizeLog2 = 7;
const unsigned int kCacheLineSize = (1 << kCacheLineSizeLog2);

I suspect the board you are using has a smaller instruction cache.

FWIW - building with snapshot off is useful as it tells us more closely where the bug is, but if you crash building snapshot - then there is a serious problem.

[edelsohn]

The most common configuration is 64 bit systems have 128 byte cache lines and 32 bit systems have 32 byte cache lines. The e500 is an oddity and has a 64 byte cache line size.

[alepesoli]

Reducing the cache line size, i finally got a clean bulding of V8 and now i'm also able to execute it with no errors; probably in the next few days i will try some other tests on it. Thank you very much for all your suggestions.

In the meanwhile, i'm also trying to compile nodejs on my e500. Currently i'm building v0.10.21-release-ppc from node github without any problems, but i have an "illegal instruction" error when i try to execute node. I don't know if i can describe here this error, so please tell me if i have to close this issue and open another one.

However this is the gdb output of my error:

(gdb) x/20i $pc-32
0x48136b0c: .long 0xc8e40003
0x48136b10: srawi r10,r3,1
0x48136b14: addi r1,r1,-8
0x48136b18: srawi r0,r10,31
0x48136b1c: stw r0,0(r1)
0x48136b20: stw r10,4(r1)
0x48136b24: .long 0xc8c10000
0x48136b28: addi r1,r1,8
=> 0x48136b2c: .long 0xfcc0369c
0x48136b30: .long 0xff873000
0x48136b34: bso- cr7,0x48136b58
0x48136b38: beq- cr7,0x48136b48
0x48136b3c: blt- cr7,0x48136b50
0x48136b40: li r3,1
0x48136b44: blr
0x48136b48: li r3,0
0x48136b4c: blr
0x48136b50: li r3,-1
0x48136b54: blr
0x48136b58: li r3,-1

After some researches, i found out that this instruction should be fcfid (used in code-stubs-ppc.cc), and from what i know e500 doesn't support instructions like fcfid, fctid or fctid.

[andrewlow]

The node build contains a copy of the v8ppc repository in deps/v8ppc

If you did not make the same cache line size change in that copy, you will be hitting exactly the same problem.

If the error is something else, please open an issue against that repository.

[alepesoli]

Ok, i will close this issue and open a new one in node repository. However i have already done that change (cache line size) in deps/v8pcc of node, so i think it's a different error.

[andrewlow]

Re-opening this issue as it clear from ibmruntimes/node#9 there are still problems with v8 on this platform.

Please run a full test of V8

$ tools/run-tests.py -j 4 --progress=dots --no-presubmit --arch-and-mode=ppc.release

You can see a successful run here: http://v8ppc.osuosl.org:8080/job/Test-PowerPC-V8

Identifying the test (or set of tests) that fails will help.

[alepesoli]

I have done the tests and the result is the following:

Running 8979 tests:
....

===
=== 1067 tests failed
=== 1061 tests CRASHED
===

As you can see a lot of tests fail/crash, and is not simple to describe all of them here.
So if you want you can download the complete log file (~300k) from the link below:

https://www.neat.it/neatcloud/public.php?service=files&file=/pesolineat/files/testlog_v8ppc_e500

[andrewlow]

Thanks for the test results. Not having the hardware makes it more difficult to address the issues for us.

If you can keep the complete log file online it'd be handy - however, I'll also cut & paste a few of the failures here so we aren't dependent on the file being available.

Command: /home/root/v8ppc-master/out/ppc.release/cctest test-api/ConversionNumber --nobreak-on-abort --nodead-code-elimination --testing_serialization_file=/home/root/v8ppc-master/out/.serdes/serdes_ConversionNumber

Command: /home/root/v8ppc-master/out/ppc.release/cctest --nocrankshaft test-api/CallICFastApi_TrivialSignature --nobreak-on-abort --nodead-code-elimination --testing_serialization_file=/home/root/v8ppc-master/out/.serdes/serdes_CallICFastApi_TrivialSignature__nocrankshaft

Command: /home/root/v8ppc-master/out/ppc.release/cctest test-api/ExternalByteArray --nobreak-on-abort --nodead-code-elimination --testing_serialization_file=/home/root/v8ppc-master/out/.serdes/serdes_ExternalByteArray

Command: /home/root/v8ppc-master/out/ppc.release/d8 --test --nobreak-on-abort --nodead-code-elimination /home/root/v8ppc-master/test/mjsunit/mjsunit.js /home/root/v8ppc-master/test/mjsunit/array-indexing.js

I'm a bit concerned that there may be an ABI issue here too - not just the missing floating point instructions. Let's take these problems down one at a time.

[alexanderturner]

Hey Andrew - I think I'm having a similar issue with a freescale MPC8541 - I would happily give you ssh access to the box if that'd help with resolving the issue.

[tunniclm]

Hi @alexanderturner, assuming you are running Linux on PowerPC, can you run the following command and provide the output?

LD_SHOW_AUXV=1 /bin/true

[andrewlow]

@alexanderturner - if you can give us access that will help speed things along, but of course you understand we have other priorities as well. Drop me an email with ssh key to access the machine and we'll see what can be done.

[jpillora]

@tunniclm I'm working on this with @alexanderturner, here's the output:

$ LD_SHOW_AUXV=1 /bin/true
AT_DCACHEBSIZE:  0x20
AT_ICACHEBSIZE:  0x20
AT_UCACHEBSIZE:  0x0
AT_SYSINFO_EHDR: 0x100000
AT_HWCAP:        booke efpsingle spe mmu ppc32
AT_PAGESZ:       4096
AT_CLKTCK:       100
AT_PHDR:         0x10000034
AT_PHENT:        32
AT_PHNUM:        9
AT_BASE:         0x48000000
AT_FLAGS:        0x0
AT_ENTRY:        0x10000c9c
AT_UID:          0
AT_EUID:         0
AT_GID:          0
AT_EGID:         0
AT_SECURE:       0
AT_EXECFN:       /bin/true
AT_PLATFORM:     ppc8540
AT_BASE_PLATFORM:ppc8540

@andrewlow, Thanks for helping out :) let us know if we should move this to its own issue as it could be unrelated...

[alepesoli]

First of all, thanks for all your help.

I want to ask your opinion about a possible temporary solution for my case. Taking into account that the performance is not important for me at the moment, i'm wondering if there is a way to emulate this instructions set which is not supported by e500, for example replacing them with C functions that does the same thing.

Do you think this could be a feasible temporary solution? And do you have any suggestion on how to do that?

[andrewlow]

@tunniclm @jpillora @alexanderturner Let's open a new issue - as Alex sent me some email and from the log it appears GCC is crashing. So not yet related to this one.

@alepesoli - sorry to clutter up your original defect. If my memory is correct, you're hitting a real problem with specific machine instructions which are not supported on the e500. Unfortunately there isn't a quick hack to resolve this, we simply need to figure out alternate implementations for the troublesome instructions. This isn't hard - but it's work. We've got an additional barrier here as we don't actually have access to the hardware, so it's going to be slow.

I'll ask we use this issue to track problems with the e500 board and the missing instructions.

[jpillora]

Created new issue here: #100

[alepesoli]

Hi Andrew, if you want we can also give you ssh access to our board.

[andrewlow]

@alepesoli - thanks for the offer. I may take you up on that as well.

I suggest that folks watching this issue also look at #100 where I've put quite a bit of data. I want to continue to treat these as different problems to have a clean separation of the data, but as we resolve one -- I strongly suspect the other will benefit.

[andrewlow]

@alepesoli thanks for access to the machine.

I wanted to capture the cpuinfo here for future reference.

$ cat /proc/cpuinfo
processor       : 0
cpu             : e500v2
clock           : 799.999992MHz
revision        : 5.1 (pvr 8021 2151)
bogomips        : 100.00
timebase        : 50000000
platform        : P1010 RDB
model           : fsl,P1010RDB
Memory          : 1024 MB

Unfortunately the login you provided doesn't appear able to build anything. The standard include file <climits.h> appears to be missing (along with other stuff I'm sure)

$ make ppc
GYP_GENERATORS=make \
build/gyp/gyp --generator-output="out" build/all.gyp \
              -Ibuild/standalone.gypi --depth=. \
              -Dv8_target_arch=ppc \
              -S.ppc  -Dv8_can_use_vfp3_instructions=true
make[1]: Entering directory `/home/v8ppc/v8ppc-master/out'
  CXX(target) /home/v8ppc/v8ppc-master/out/ppc.release/obj.target/preparser_lib/src/allocation.o
In file included from ../src/allocation.cc:33:0:
../src/utils.h:33:19: fatal error: climits: No such file or directory
compilation terminated.
make[1]: *** [/home/v8ppc/v8ppc-master/out/ppc.release/obj.target/preparser_lib/src/allocation.o] Error 1
make[1]: Leaving directory `/home/v8ppc/v8ppc-master/out'
make: *** [ppc.release] Error 2

Are the standard files installed in a non-standard location? Is there some sort of setup I need to do?

[alepesoli]

You are right, i'm really sorry, this board has an embedded version of linux, so some files are not exactly in the "usual" place. Exporting CXXFLAGS with the following command should fix the problem and let you complete the building.

export CXXFLAGS="-I/usr/include/c++ -I/usr/include/c++/powerpc-fsl_networking-linux-gnuspe"

[andrewlow]

SSL certificates seem cranky too - making git complain.

Easy to fix

$ export GIT_SSL_NO_VERIFY=true
$ git clone https://github.com/andrewlow/v8ppc.git

[andrewlow]

The machine doesn't have svn installed, that'd be handy.

Building the PPC codebase in 'simulation mode' let's us avoid some of the incompatibility issues - it'll be slow, but should work

make ppc nativesim=true

Testing can be accomplished by

nohup tools/run-tests.py --progress=dots --no-presubmit --arch-and-mode=ppc.release  | tee test.log

[andrewlow]

There are errors in the simulated build, but they also appear when built on a Power7 machine - so they are just simple defects vs. problems that are related to the platform.

So we can build a shared library version of v8

make ppc nativesim=true library=shared

Then checkout 'node' from https://github.com/andrewlow/node and build/link to it

./configure --shared-v8 --shared-v8-includes ~/v8ppc/include --shared-v8-libpath ~/v8ppc/out/ppc.release/lib.target
make

[edit - had to remove relative paths]

[andrewlow]

$ export LD_LIBRARY_PATH=~/v8ppc/out/ppc.release/lib.target/
$ ./node
> console.log("hello world");
hello world
undefined
> 

Ta-da.

So let's not forget, this is a simulated version, so performance is going to be a slow. However, this should allow people to prototype. There are also bugs in the simulation which need to be fixed up (these are specific to the simulator being built on the native platform, so thus related to calling convention in some way).

[alepesoli]

That's great! But i don't understand the difference between the "simulated" version and the "standard" one, could you please explain me how PPC codeabase is simulated.

[andrewlow]

The V8 implementation supports both a 'native' and a 'simulated' mode. Primarily this makes sense when dealing with less friendly embedded platforms (think ARM / Cell Phones).

Thus, even with the PPC version, we can build it on an Intel based system. The code then generates PPC instructions into memory of your Intel machine - then at runtime it simulates a PPC CPU to run that code.

What I've done here is built the simulated version of PPC on a PPC machine via the nativesim=true flag. Yes, this is a bit weird, but it allows us to run something without actually fixing the core issue. With the ability to run Node.js on this (not yet supported) version of PPC, we can explore any other incompatibilities.

Again, it's slow. The time to run "npm -v" clocks in around 8 or 9 seconds. (but hello world is a more reasonable sub-second time). We won't win any benchmarks in this mode.

[andrewlow]

These are unofficial measurements - conducted on a multi-user dev machine which itself is virtualized. Still it gives a sense of the performance difference between 'native' and 'simulated'

-bash-4.1$ cat test.js
var time = process.hrtime();
eval(process.argv[2] || '1+1');
var diff = process.hrtime(time);
console.log('benchmark took %d ms', (diff[0] * 1e9 + diff[1])/1e6);

Native performance

-bash-4.1$ node test.js 'var i = 100000; while(i--){}'
benchmark took 36.624761 ms

Simulated performance

-bash-4.1$ ./node test.js 'var i = 100000; while(i--){}'
benchmark took 359.965012 ms

Again, I will caution that you can't simply calculate a factor an apply it to all operations based on this quick test. Also, as you can see the benchmark itself is highly compute intensive.

These were run on
cpu : POWER7 (architected), altivec supported
clock : 4004.000000MHz

[dpasupathi]

Hi,

I tried to load ppc version of node (v0.12.4) and i am still seeing the Illegal instruction error while executing the node.

Please let me know if this issue is resolved or is there any workaround? My cpuinfo is as follows:

cat /proc/cpuinfo
processor : 0
cpu : e500mc
clock : 1499.400000MHz
revision : 2.2 (pvr 8023 0022)
bogomips : 46.85

processor : 1
cpu : e500mc
clock : 1499.400000MHz
revision : 2.2 (pvr 8023 0022)
bogomips : 46.85

total bogomips : 93.71
timebase : 23428125
platform : SILKWORM PB HV
chipset : P4040
Vendor : Freescale Semiconductor
Memory : 4096 MB

LD_SHOW_AUXV=1 /bin/true
AT_DCACHEBSIZE: 0x40
AT_ICACHEBSIZE: 0x40
AT_UCACHEBSIZE: 0x0
AT_SYSINFO_EHDR: 0x100000
AT_HWCAP: booke mmu fpu ppc32
AT_PAGESZ: 4096
AT_CLKTCK: 100
AT_PHDR: 0x10000034
AT_PHENT: 32
AT_PHNUM: 8
AT_BASE: 0x18000000
AT_FLAGS: 0x0
AT_ENTRY: 0x10000a50
AT_UID: 0
AT_EUID: 0
AT_GID: 0
AT_EGID: 0
AT_SECURE: 0
AT_??? (0x19): 0x3fb9e7fc
AT_??? (0x1f): 0x3fb9eff2
AT_PLATFORM: ppce500mc
AT_??? (0x18): 0x3fb9e80c

[turneyj]

I believe the e500mc only lacks support for two PPC floating point instructions v8ppc uses - fcfid (convert double int to float) and fctidz(convert float to double int). A much closer "fit" than the e500v2 which has a completely different floating point core. I was looking at the e500v2 port and need to push my changes for the work done so far on that. But the e500mc and subsequent Freescale cores should be easier to support.

[dpasupathi]

Hi Turneyj,

Thanks for the quick response. Is there any milestone targeted for getting the fixes for e500mc ??

[dpasupathi]

I am not an expertise in this assembler code. Does someone have a patch so that I can try make nodejs work on my powerpc environment?

[sbabic]

Hi Gary (Turneyj),

I am also facing the same issues with a P2020. Before starting to check deeper, I am kindly asking you which is your current status and if you are still working on this port. I have checked your repository, but I have only seen the fork from Andrew's 3.14-ppc and no commits for e500v2. Do you plan to push your current changes ? It is surely a good starting point to go on.

[readysteadygo2006]

Hi,

I'm trying to get node working on ppc(e500v2). I've read through the
threads, I have a grasp of why a compilation which uses the HW FPU
on the e500v2 will not work, however I don't understand why a compilation
which uses -msoft-float woud have a problem.
Can someone explain this?
It would be much appreciated!

[andrewlow]

Sigh.. answered this query here #119 (comment)