Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xenia for Mac? #596

Open
1NEGROUP opened this issue Jul 3, 2016 · 85 comments
Open

Xenia for Mac? #596

1NEGROUP opened this issue Jul 3, 2016 · 85 comments

Comments

@1NEGROUP
Copy link

1NEGROUP commented Jul 3, 2016

Is there a way to Compile for Mac?

@Razzile
Copy link
Member

Razzile commented Jul 3, 2016

macOS doesn't meet quite a lot of the requirements for xenia.

Until it gets an updated openGL driver to support 4.5 or vulkan support (unlikely right now) xenia cannot run on mac.

Your best bet would be to dual boot and run xenia on windows.

@Razzile
Copy link
Member

Razzile commented Jul 29, 2016

even with that, there is still a huge amount of porting work needed. For example xenia currently relies on quite a few windows API calls

@MrSapps
Copy link

MrSapps commented Jul 29, 2016

moltenvk isn't even free..

@Razzile
Copy link
Member

Razzile commented Jul 29, 2016

^ that too

@MrSapps
Copy link

MrSapps commented Jul 29, 2016

seems unlikely

@MightyElemental
Copy link

xbox means DirectX box. Macs do not have DirectX so I doubt there will ever be a version for Mac.

@Razzile
Copy link
Member

Razzile commented Oct 15, 2016

@Gamedev1909 Are you sure about that?

@ghost
Copy link

ghost commented Oct 15, 2016

@MightyElemental but they can write OpenGL/Vulkan driver so it could work on Mac, it is possible, but not worth at actual state of xenia.

@gibbed
Copy link
Member

gibbed commented Oct 15, 2016

There is nothing stopping a OSX build from existing, other than a contributor who would make the necessary effort to get it building on OSX. The main contributors don't develop on OSX.

@Razzile
Copy link
Member

Razzile commented Oct 15, 2016

@gibbed the problem with OS X (at least for now) is awful graphics support. No vulkan and the max OpenGL version is OpenGL 4.2.

If mac gets proper graphics drivers I'd be more than happy to attempt to port the rest

@Fraxul
Copy link
Contributor

Fraxul commented Oct 15, 2016

Not even 4.2, it's 4.1 on the latest Mac OS. Being a graphics developer on Mac OS is a hard life.

Metal definitely has the features to host Xenia, but it'd be better to get the graphics subsystem closer to feature-complete/working before trying to port to yet another hardware backend.

@Margen67
Copy link
Member

You have no idea what you're talking about. Just stop.

@hlide
Copy link
Contributor

hlide commented Oct 16, 2016

So just contribute to porting Xenia upon OSX. Less wording, more coding.

@D1SMBD1D
Copy link

C'mon, no one has worked on the emu in a month and you want someone to rewrite craploads of code so it runs on a mac?

@petmac
Copy link

petmac commented Apr 3, 2017

Does Xenia definitely require features from OpenGL 4.5, or is that the minimum requirement because it's more convenient?

Maybe it would be possible to support 4.1 or 4.2 with some smaller scale reworking.

@Margen67
Copy link
Member

Margen67 commented Apr 3, 2017

@petmac OpenGL is deprecated.

@Razzile
Copy link
Member

Razzile commented Feb 26, 2018

it seems MoltenVK went open source https://github.com/KhronosGroup/MoltenVK
I am not a graphics dev so i have no idea how useful or feature-complete it is for xenia but i guess it's a glimmer of hope for those with a mac

@0x8080
Copy link
Contributor

0x8080 commented Feb 26, 2018

  1. Fuck Apple and their refusal to use any industry standards and their shitty closed ecosystem
  2. OSX does not natively have Vulkan (MoltenVK is not native and it's not free) or anything past OpenGL 4.1 (you can find Drivers for Vulkan and OpenGL 4.5 on fucking Windows XP)
  3. Get a decent OS

Correction: MoltenVK went Opensource

@0x8080
Copy link
Contributor

0x8080 commented Feb 26, 2018

@Gamedev1909 what's the difference between a peice of shit and a peice of shit 2?
In this case not much since Apple barely do anything new with newer Mac OS releases.
Its just a name and a number, they're both shit.

@0x8080
Copy link
Contributor

0x8080 commented Feb 26, 2018

They added fan control and overclocking...

When Windows and Linux had this for years.

Also I do have macOS and OSX and it was a painful experience for me.

Drivers barely worked, UI was over simplified to the point where navigating system files was a pain and general usability is horrible for a power user like me.

This isn't even bringing up how much of a pain in the ass it is to setup MacOS on non-Apple hardware or even attempting to fix issues with OSX

@tetration
Copy link

tetration commented Feb 26, 2018 via email

@Razzile
Copy link
Member

Razzile commented Feb 26, 2018

this thread got really off topic really quickly...
I was only pointing out moltenvk is now available for future reference.

@sekkit
Copy link

sekkit commented May 29, 2018

Since Hackintosh can achieve very high performance hardware spec, when xenia is feature complete, I strongly suggest porting it to MacOS also expecting that I can offer some practical work.

@kvark
Copy link

kvark commented Aug 26, 2018

Metal has no way to swizzle so its semi pointless unless in 7 days metal 3 has swizzle ability

This is unfortunately true. RPCS3 was able to work around this in RPCS3/rpcs3#5024

@Triang3l
Copy link
Member

It also doesn't support geometry shaders, which are needed for clear rectangles, and some significant memory usage optimization features used in the new Xenia renderer — sparse resources and heap aliasing.

@kvark
Copy link

kvark commented Aug 26, 2018

@Triang3l oh, interesting! I'd like to know more.
How are the geometry shaders used for clears? Are sparse resources and heap aliasing non-optional in your Vulkan renderer?

@Triang3l
Copy link
Member

@kvark Xenos clear render targets by drawing a large rectangle, however, only 3 vertices are submitted, so we need to reconstruct another one in a geometry shader from the 3 vertices we have, after the vertex shader is done.

Regarding the memory-saving features: I was talking about my Direct3D 12 renderer rewrite, but this applies to Vulkan as well. They are optional, but definitely help reduce video memory usage.

I'm using a 512 MB buffer to emulate physical memory layout for texture untiling/tiling and vertex fetch because it increases performance (no need to maintain vertex caches) and accuracy (allows for weird render to texture cases, including horizontal predicated tiling). However, games don't necessarily use all those 512 megabytes for textures and vertex buffers, so it's better to allocate that buffer in smaller parts.

Resource aliasing is great for render targets since they're transient on the Xbox 360 due to EDRAM. Without aliasing of heap contents, memory would have to be allocated for every used size and format combination, but arbitrary allocation in heaps lets us reuse the memory backing render targets and simply create views of it with different sizes and formats.

@Juesto
Copy link

Juesto commented Aug 26, 2018

What features does Xenia depend on? anything heavily used?
if someone goes through the hassle of reducing ogl support and falling back features it would run but at less performance. blame apple really
also moltenvk isn't as ideal since the limitations imposed by apple still makes some things impossible.
yet there's no point, especially since Mojave is removing OpenGL.

@petmac
Copy link

petmac commented Aug 26, 2018

OpenGL is being deprecated, not removed... Yet.

@iMacker2020
Copy link

I wanted to share this information I found: https://gamefromscratch.com/cross-platform-opengl-alternatives/. It list libraries that actually sit on top of graphics libraries like Metal, OpenGL, DirectX, and Vulkan. This means writing code once in say bgfx can be used on all supported graphics libraries (cross platform).

The ones that impressed me were:

@Triang3l
Copy link
Member

Triang3l commented Mar 29, 2022

I don't think there's any need for us to give up the ability to do per-API optimizations/fallbacks and to use API-specific extensions and instead adapt to the architecture of somebody's wrapper. On Vulkan, we already have to handle way more device features/limits cases (as we're ultimately targeting not only the PC implementations of Vulkan, but the entire set of Vulkan 1.0-compliant devices, as well as portability subset ones) than on Direct3D in pretty low-level and careful ways, this is basically the primary difficulty of porting to Vulkan.

@Triang3l
Copy link
Member

Triang3l commented Mar 29, 2022

Metal, in particular, needs certain changes specific to it, such as working around the inability to disable primitive restart.

@iMacker2020
Copy link

Metal, in particular, needs certain changes specific to it, such as working around the inability to disable primitive restart.

So you think the best thing to do is to replace Vulkan with Metal on Mac OS?

@Triang3l
Copy link
Member

Triang3l commented Mar 29, 2022

Metal, in particular, needs certain changes specific to it, such as working around the inability to disable primitive restart.

So you think the best thing to do is to replace Vulkan with Metal on Mac OS?

MoltenVK would likely be more of a limitation on macOS, and would probably cause a lot of noise — there are many things that are specific to Vulkan, but not relevant to Metal, and there are parts that are specific to Metal that may need attention. Plus, the current version of MoltenVK translates SPIR-V shader bytecode into the Metal Shading Language — essentially C++ — rather than to the LLVM-ish shader IR directly. This will likely make pipeline state object compilation (mid-game when entering new scenes, at launch when loading the pipeline state cache) around an order of magnitude longer. I remember the early days of working on the Direct3D 12 backend — initially I was translating shaders into HLSL rather than DXBC (which even still has some overhead compared to the more modern, but slightly less widely supported — not on old Intel GPUs and the UWP on the Xbox One — DXIL), and compiling the pipeline state objects for the first frame of Halo 3 took 2 minutes on my PC, rather than a few seconds. When Xenia used GLSL, the situation was roughly the same as well.

@iMacker2020
Copy link

Ok so the path to Mac OS compatibility is:

  • Translate PowerPC machine code to ARM64 using xbyak for aarch64
  • Implement a DirectX 9 to Metal translation layer

@Triang3l
Copy link
Member

Triang3l commented Mar 29, 2022

(not DirectX 9 to be more precise, rather, ATI R400 registers and PM4 packets)

Regarding PowerPC translation, I'm not sure if we should even bother creating a new backend for the current IR, or rather, work on it as part of a large-scale rework of the entire recompiler architecture.

Also, all the Linux requirements (and potentially on the very low system level, some parts will be different than on Linux even considering that macOS is not running on the Linux kernel), primarily on the Xbox 360 kernel reimplementation side, still apply, as well as macOS implementations of UI, input and audio interfaces.

@iMacker2020
Copy link

Well the PowerPC translation issue might not be so important right now. Rosetta 2 can probably handle the current x86 backend. So things would be like this: PowerPC -> xbyak x86_64 -> Rosetta 2 -> M1 CPU. That is a lot of translation but I do believe it can be done. The Xbox 360's hardware is around 17 years old now. Our modern hardware should be able to handle the translation layers and keep up with the games. I do hope to make things simplified in the future.

What was your idea for a "rework of the recompiler architecture"?

I did some reading here #1430. Looks like sound is implemented using SDL2. Now that I think about it almost everything could be implemented on top of SDL and run on both Mac OS and Windows.

@Triang3l
Copy link
Member

@iMacker2020

What was your idea for a "rework of the recompiler architecture"?

I'm thinking about making our HIR more PowerPC-like (basically PowerPC code in SSA form and with structured control flow), but with more advanced optimizations (including instruction scheduling) for the host architecture, involving host-specific intermediate representations in addition to the PowerPC frontend one, instead of the current model where we basically have some parts of a compiler on the level of the HIR (which is quite abstract and arbitrary, and pretty far from both PowerPC and x86), but the host backends act roughly as simple translators of the HIR instead. I've written various notes in #2002 (comment) regarding this.

@Triang3l
Copy link
Member

With 166be46, almost all of our built-in "Xenia Shading Language" shaders can now be compiled as MSL.

The exception is the FidelityFX ones, as they require writing the FidelityFX platform definitions that currently exist only for GLSL and HLSL, and also wrapping the whole shader in a class storing references to the bindings as fields initialized in the constructor called via an entry point thunk, since Metal resource bindings, unlike GLSL/HLSL ones, are not in the global scope, but rather, they're entry point arguments that must be passed to callees explicitly if needed (or, references to them must be stored in some object that is also passed somehow, including via this) — but the FidelityFX texture fetch callbacks don't offer a way to pass "user data" that includes the texture object to them. We can write pure Metal versions of those shaders, but to avoid having to create exceptions in xb buildshaders, store them as include files rather than as source files, and #include them instead of the common code in #if XESL_LANGUAGE_MSL in the actual XeSL files passed to compilers.

Note that because of our custom .xesl extension, the -x metal option must be passed to the metal compiler before the input argument so the source files are interpreted as MSL.

@kylejericson

This comment was marked as off-topic.

@picajoso
Copy link

I wonder if the recent announcement by Apple in the WWDC'23 changes things with the Game Porting Tookit:

https://developer.apple.com/videos/play/wwdc2023/10123/

Seems promising!

@mavethee
Copy link

I wonder if the recent announcement by Apple in the WWDC'23 changes things with the Game Porting Tookit:

https://developer.apple.com/videos/play/wwdc2023/10123/

Seems promising!

In some context probably, we definitely can't test how Xenia would ran on a Mac as it requires AVX instruction set and Rosetta can't interpret those, tested it out a few days after WWDC and yeah, expected result.

image

@programmingkidx
Copy link

Now I'm thinking about trying to find a way around this problem. One thing that comes to mind is rewrite the AVX code to use something more portable like a library. This library sounds promising: https://ermig1979.github.io/Simd/

@Wunkolo
Copy link
Contributor

Wunkolo commented Jun 20, 2023

Xenia actually emits AVX code during translation of PowerPC instructions at runtime. Implementing a non-AVX version would be very non-trivial and quite an overwhelming revisiting of all of our emitted assembly.

@mavethee
Copy link

For anyone coming here, as of macOS 15 Beta 1, Rosetta 2 doesn't expose any information about supporting either AVX or AVX2 which Xenia or RPCS3 check for, thats why it won't work for now, whatever their approach to this is, it's a mess now.
CleanShot 2024-06-12 at 18 11 28@2x

@Wunkolo
Copy link
Contributor

Wunkolo commented Jun 12, 2024

I only seem to see AVX2 mentioned in relation to GPTK2 and not Rosetta generally.
https://developer.apple.com/games/game-porting-toolkit/
image

I should note though that I have implemented an ARM64 backend for Xenia that is just waiting to be merged at the moment, so the need for emulation through Rosetta may not be necessary. I have an M2 Mac Mini I could do Mac testing on, but I don't think there is any build-infrastructure in-place for Xenia to compile on Mac at the moment. The CPU-backend is ready to go though once that comes online.
#2259

@kode54
Copy link

kode54 commented Jun 13, 2024

From someone I know who tested it, it's not exposed by the kernel yet, so you can't run macOS x86_64h architecture binaries yet. But you can detect it at runtime with the cpuid instruction.

Edit: I have not verified this myself, as I have not installed the beta yet.

@mavethee
Copy link

mavethee commented Jun 13, 2024

I only seem to see AVX2 mentioned in relation to GPTK2 and not Rosetta generally.

AVX2 needs AVX1 to work, that's kinda obvious.

What's funny some of AVX1 feature set work, tested in RPCS3 with LLVM CPU set to AVX1 CPU runs with slightly better perfs, but setting AVX2 CPU crashes.

General rule seems to be some stuff will work unless it won't actually check for either AVX1 or 2 support.

Yes, Apple mentions AVX2 in docs because end users already confuse it anyways.

Thanks, Apple I guess 🎉

@shinra-electric
Copy link

shinra-electric commented Jun 13, 2024

I forked Xenia and merged the arm64 backend PR into it, and came across several issues building on macOS:

  • Ran ./xenia-build setup which failed.
    • Issue 1: Premake is preset to try to build x86. I built it manually and put it in xenia/tools/build/bin
  • Ran ./xenia-build build --config=release which failed
    • Issue 2: all the generated xcode projects had an incorrect value of ARM64 as their target architecture. It should be a lowercase arm64. I just opened the xcworkspace in xcode and changed it to $(ARCHS_STANDARD) on all targets
  • Built the xenia-app target
    • Issue 3: No Info.plist file. Changed the Generate Info.plist file build option to yes in the xenia-app target
  • Built the xenia-app target
    • Issue 4: missing arm64_neon.h file

Edit:
Here is a script for building Premake if anyone wants to try to get any further
Premake.sh.zip

@hackgrid
Copy link

It works on WINE / GPTK2 now:
https://www.reddit.com/r/xenia/comments/1ddnrvh/comment/l861md6/
https://www.youtube.com/watch?v=ELug8rz1rBg

Native build would be way better of course :)

@mavethee
Copy link

Native build would be way better of course :)

Thats long way to go, even RPCS3 isn't native.
Ideal goal for now would be AVX exposure to pass the check.

As of Beta 2, Rosetta 2 was updated but still reports same feature set.

Features: AESNI,CLMUL,CMOV,CMPXCHG8,CX16,FXSR,FXSROPT,LAHF,MMX,NX,OSXSAVE,POPCNT,RDTSCP,SSE,SSE2,SSE3,SSE4,SSE42,SSSE3,SYSCALL,SYSEE,X87,XSAVE

Their feedback answers towards this issue are somewhat funny and concerning at once.

@hackgrid
Copy link

hackgrid commented Jun 25, 2024 via email

@mavethee
Copy link

mavethee commented Jun 26, 2024

Are you sure? You can compile rpcs3 native for aarch64 Linux, have not tried for MacOS though yet...

Yes, you can but it doesn't mean it will work just as good as x86_64 release, especially in macOS case.

Linux at least have way better support than macOS does.

See: RPCS3/rpcs3#15567

But that's getting offtopic a bit.

@mavethee
Copy link

mavethee commented Jul 11, 2024

As of macOS 15 Beta 3, thats the funniest shit I've ever seen:
image

But to be fair, regardless of CPU-Z through Wine report and recently introduced AVX exposure variable, it's just can't detect it lol.

Using arch -x86_64 cpuid (CPUID installed from Homebrew)

Features: AESNI,CLMUL,CMOV,CMPXCHG8,CX16,FXSR,FXSROPT,LAHF,MMX,NX,OSXSAVE,POPCNT,RDTSCP,SSE,SSE2,SSE3,SSE4,SSE42,SSSE3,SYSCALL,SYSEE,X87,XSAVE

@udance4ever
Copy link

It works on WINE / GPTK2 now: https://www.reddit.com/r/xenia/comments/1ddnrvh/comment/l861md6/ https://www.youtube.com/watch?v=ELug8rz1rBg

anyone have this working in the GA release of macOS Sequoia?

I had it working in beta a few weeks ago and it was pretty impressive to see it even running through so many layers of translation. One of the titles I own is just a simple XBLA puzzle game and it worked well enough to take the game on the go.

I updated to macOS Sequoia and built xenia_canary from from source with the AVX check patched out and now it just crashes.

I'm under the assumption that GA release of Rosetta 2 has the "experimental" AVX2 bits patched out. Anyone else confirm this or otherwise?

@mavethee
Copy link

mavethee commented Sep 19, 2024

It works on WINE / GPTK2 now: https://www.reddit.com/r/xenia/comments/1ddnrvh/comment/l861md6/ https://www.youtube.com/watch?v=ELug8rz1rBg

anyone have this working in the GA release of macOS Sequoia

I had it working in beta a few weeks ago and it was pretty impressive to see it even running through so many layers of translation. One of the titles I own is just a simple XBLA puzzle game and it worked well enough to take the game on the go.

I updated to macOS Sequoia and built xenia_canary from from source with the AVX check patched out and now it just crashes.

I'm under the assumption that GA release of Rosetta 2 has the "experimental" AVX2 bits patched out. Anyone else confirm this or otherwise?

You are fully aware patching the check is not a solution, it's just Apple fuckup that they fail to recognize or I have luck to lame employees jumping on the issue. Current AVX implementation still has issues even with their "promise".

CleanShot 2024-09-19 at 08 46 29@2x

I like how when I covered Mac app issue, they said ItS fOr WiNE oNlYy forgetting I've mentioned also running Windows exec (Xenia in question ofc) through Wine ffs.
Yeaaaah....

@SherlockWh0
Copy link

It works on WINE / GPTK2 now: https://www.reddit.com/r/xenia/comments/1ddnrvh/comment/l861md6/ https://www.youtube.com/watch?v=ELug8rz1rBg

anyone have this working in the GA release of macOS Sequoia?

I had it working in beta a few weeks ago and it was pretty impressive to see it even running through so many layers of translation. One of the titles I own is just a simple XBLA puzzle game and it worked well enough to take the game on the go.

I updated to macOS Sequoia and built xenia_canary from from source with the AVX check patched out and now it just crashes.

I'm under the assumption that GA release of Rosetta 2 has the "experimental" AVX2 bits patched out. Anyone else confirm this or otherwise?

for xenia to run in gptk/wine on sequoia, you need an edited build with avx detection patched out, as macos does not advertise to the programs that avx is supported, even though it is supported.

@mavethee
Copy link

Well Apple claims they do but it's hard to speak to them through feedback when they treat you like an idiot and when you loose temper they just dont respond to your tickets at all lmfao.

So yeah it works but implementation by either side is so shitty that it doesn't if that makes sense.
After all, as e.g Rift Apart instead of crashing, boots normally as game doesn't do a detection check on launch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests