How I implemented Steamworks API in two EasyRPG Player based games at 2018 and not died in the attempt: A chaotic testament.

 Hi! Time of a very large post!

    This is a topic I wanted to talk about years ago. And you might ask: Then why did you didn’t? Easy answer: I didn’t have a place to write crap like this. (Well, technically speaking I always had this blog, but it was forgotten. Read the past post for that history). And since there are people which asked about this topic from time to time in assorted channels, then I decided is time to write about it.

Okay, let’s start.

    One of the atypical personal projects I have done in the past years was helping Netherware Entertainment with publishing 2 games of theirs: Desolate City: The Blooding Dawn Enhanced Edition and Super Crystal Hunter in that popular PC platform that we all know by its name Steam. Easy task, no?

Wait wait… Netherware? Desolate City? Super Crystal whathever? Explain yourself!

Okay okay, calm down buddy! I will explain it!

Let’s start explaining what Netherware Entertainment is. Long history short: An indie videogame developer company ran by a group of talented people which uses RPG Maker 2003 as a main weapon of choice! Lead by CesarRP, they released a nice bunch of games, which you can take a look at their website. They also make a lot of great videos/podcasts (in spanish) about RPG Maker game developing and publishing. Go take a look a their social media!

               The roots of this company born many years ago, when this was just all fun and games (?) when a bunch of people made a GD (A short of Grupo Desarrollo … or Development group in the RPG Maker scene slang), called PARPG. I participated there sometimes trying to giving my knowledge regarding different things such as chicken-based instrument melodies, tools&patches advisory or even avant-garde trailers.

(Yeah, that happened. And I believe that legendary cap (from Tahiti) still existed somewhere in my wardrobe. )

    The years passed, people go, people go away (so I do), etc. But CesarRP, the head on that group took that PARPG and converted in something way far more serious that’s what we call now Netwherware Entertainment. And that’s the history of!

    And yeah, Desolate City: The Blooding Dawn Enhanced Edition and Super Crystal Hunter were two awesome games from this game house. 

RPG Maker 2000/2003 and Steamworks: Not a nice mixture.

    Something big were being cooked at Netherware Entertainment at that time: Desolate City was about to be released in Steam with a brand new Enhanced Edition with plenty of new stuff compared with the early version of the called The Atrezzo years ago at still PARPG eras. Wow! And even with a publisher in the back.

    Of course, dealing with publishers and platforms such like Steam means it requires plenty of extra stuff. Let’s say: Integration with the platform menus (Steam Overlay), Achievements, Embedded Multilanguage support, Multiplatform, etc etc etc etc… Of course, Steam does not require to implement all of those but it definitively a nice-to-have! And seems like at those years there are already people and companies with experience dealing with all of that with NodeJS based RPG Makers (MV/MZ) and Ruby based ones (XP/VX(a)). But what about 2000/2003? 

    Seems there were attempts to publish RPG Maker 2003 games at the time (2018 approximately ), but by putting all those extra features away; in other words: Windows only, no achievements, no overlay, just one language, etc. (Years after there were a increased number of games using it but … after). Other popular option was just switching to use more modern versions of RPG Maker. It was a decision for example, made at, Yume Nikki or One Shot, which was not very welcomed by the fans of their original versions.

So… what was left for Desolate City? Of course, call EN.I back!

(Diablos, Steam API didn’t run with 2k n 2k3, and it’s not possible to add achievements and whatnot). 

    It all started with a conversation with CesarRP regarding the golden ages and future plans of their projects and Netherware itself. I will not post the 32km long conversation thread we had for the whole thing (because 1. It’s boring. 2. Just scrolling back to 2018 for getting that screenshot my browser almost crashed twice for running out of memory :P ). He is telling me about dealing with exactly the same I was told you some paragraphs upper. At that time I remembered was a bit bored with my daily job at that time, and that sounded like a nice challenge to have. So I agreed to investigate about! 

First experiments with RPG_RT

    My first attempt to solving an issue like this (at that time we are dealing with multilanguage support) is trying to solve that thing the way I was used to know at golden eras of RPG Maker: Patches, patches and more patches. I was thinking about callbacks with Ineluki KeyPatch (Patch that replaces Harmony Audio Engine with a upper layer that allows you to invoke executables when playing specific audios) to make callbacks to SteamAPI and do things, or even that DynRPG (which you can code Plugins in C/C++) for the same…

… Or what about doing my own patches for that?

(Well… that idea did not evolved in something more, but at least, the remains of it were recycled and made this tool https://github.com/elsemieni/RPGMaker2k3VarInspector . )

So, as apart from that, CesarRP was also dealing with Unicode issues (remember, RPG_RT does not support that; you need an special crafted RPG_RT hack for each locale you want to), remembered that EasyRPG was more and more mature in that era (reaching it’s 5.4 release at that time). Still having a bunch of nasty bugs at that time, but It was able to run games in a playable state at a really nice percentage (with some aesthetic sacrifices sometimes).

So… we had a discussion with CesarRP: How about selling games with EasyRPG Player? GPLv3 license does not forbid from doing that (as long as we publish any modified source code with it if we do so), it have an awesome (but at the time a bit mysterious in terms of how-to-properly-use-it) Unicode support and lots of mini extra features we can use! Yeah, as I said, it had some nasty bugs, but we agreed we can make workarounds inside the game for making the game working properly. 

EasyRPG Player, GPLv3 and Steamshim

Ok, so for making a prototype to make it work I just need to plug Steamworks SDK into EasyRPG Player source code, compiling it an… wait wait WAAAAIT! There is an issue, and not a technical one. You can’t do that! The reason is simple: EasyRPG is GPLv3 which requires the code should be shared with the binary, and forbids linking against proprietary libraries (with some exceptions if those libraries are crucial for operating system… but that’s not the ideal for a free-as-in-freedom fanatic). And of course, SteamSDK is a fully proprietary code with no source code available plus some clauses of privacy (even if it’s super easily downloadable with tons of contents available around the internet). In short words: It’s illegal linking EasyRPG Player with Steamworks SDK… Is a very similar reason why EasyRPG Player can’t be used in gaming consoles or some other privative environments. A solution for that is switching Player’s license to something more permissive such as MIT; and there’s a 32km long discussion regarding that for years! https://github.com/EasyRPG/Player/issues/167

So... The end? Nah.

Luckly, I was not the only one in this same sort of issue. Someone called icculus, which is the guy behind SDL  created a clever but shady solution for it called Steamshim  which basically is a bridge between your GPL executable and another isolated executable containing the privative code that links with the Steamworks API. That bridge is not connected by “code” but with system pipes, making this ting barely legal. Ugly/shady? Yes. It works? Yes.  It is legal? Barely, but yes.

So, I took that Steamshim and immediately tried to implement the examples into a custom artisanal-crafted EasyRPG build. After dealing with trying to build it in Windows with AutoTools and MinGW (NOT RECOMMENDED XDD… time after I learnt that I should build EasyRPG Players in Windows using CMake/VisualStudio instead ) I got a functional basic setup.

(Had to say … I was only able to make dynamic builds with MinGW and I think I was one of the few people I was able to build Player with MinGW… Also, if you recognize that folder called _Grise_ you will know that I used The Bad Guy  as a testing game… which I believe it one of the very first games that EasyRPG was able to play in it’s very early versions 😊 )

The concept was easy: Talk with a Chara and it will trigger an achievement (we mimic that we are playing Postal by putting it’s steamappid.txt there), and testing overlay included. Worked flawlessly.

(Yay!)

So, the possibility is now real. So… let’s work! We have a game to “port”. 

Putting all together.

For triggering all of Steam stuff from RPG Maker, I choose an old-school approach. In old days, when you are using patches, you usually listened all sorts of commands in different combinations. For example, in BetterAEP (a patch by CherryDT that allows skip the title screen in old RPG_RTs with extra features for exiting and triggering the Load Menu), for calling the Load Menu, you needed to put certain values in certain variables and then call “End Event Processing”.

Here I choose a similar approach: Each achievement will have an internal name something such as ARCHIVEMENT_NUMBER_X where X is an arbitrary number from 0 to … n. (whatever the number of achievements the game had, can’t recall now). So, I just made something like this:

 

Put a variable with value 1 (for example), another variable with certain value (to replace End Event Processing default behavior) and called End Event Processing. Then modified event interpreter source code to evaluate that and then trigger the achievement ARCHIVEMENT_NUMBER_1 (as the variable is set at 1). In that way CesarRP was able to trigger achievements in any place he wanted to, using convenient common events for that. 

(Achievements working on Desolate City. It was emotional... regardless the fact it is a crappy debug screen in a debug conditions. But it was emotional anways...) 

For achievements based on metrics, the logic was very, very similar. Just have special commands to set the desired metric and the desired value (for example one variable for set bullets, and other variable to set the number of bullets shoot) and End Event Processing for sending that to Steam. Then steam itself will trigger an achievement if applicable. 

(Some achievements were… a bit complex logic to be triggered as you can see)

Another interesting thing that Steamworks have about is regarding how it handle game’s data internally. Without digging so deep, it uses a “Repository” system, which separates the whole game data in different “modules”, each one having different attributes (OS, language, DLC, etc). So, by providing different repositories you can have different game setups (For example: Base game data + Windows Binaries + DLC).

With that in mind we can play with that for adding interesting features using that Repository feature:

• Auto language detection: For example, if Spanish was set, it will download a Repository which contains a Map0004.lmt (for example) which have an automatic a event that set a Language variable to 2 (for example) then teleporting to the intro.  Of course you can, after that, set any variable you want.

• DLC support: For example, If DLC was purchased, it will download a Repository which contains extra maps, extra data files and a Map0005.lmt (for example) have an automatic event that set a DLC variable to 2 (for example) then teleporting to the intro.  Then it will check DLC variable for enabling or disabling certain features.

Cloud saving? Just tell Steam that it should handle about .lsd files and that’s it. Now we have cross cloud storage!

Overlay was not much science. With just integrating Steamworks with Player you already have it. However many Steam buttons conflicts with mapped button of EasyRPG player, sooo… We did not have more option rather than re-map EasyRPG buttons to something else. It was easy, as we are already modifying the source code.

Do I miss something here? Yes! Cross-OS compatibility! But let me make a pause first as it is 2AM, so I will continue tomorrow to write that chapter. (Yes, this post was made in 2 days).

Desolate city, available in your toaster and in your staubsauger 

One of the nice features of EasyRPG Player of course, is cross-compatibility, and Steam welcomes that (at least for Windows, GNU-Linux (ahem* SteamOS/SteamDeck nowadays) and MacOS ). And since we are using it, then why not we can offer that capability (even if probably, most players will come from Windows OS... or maybe SteamDeck probably).

As I told you some paragraphs before, Steam, by it’s repository modular system, you can switch OS binaries by simply specifying the OS. So we can make a customized game setup using the correct binaries for each platform. But there is just a tiny problem: Remember our player is customized, right? We can’t just take the binaries from  https://easyrpg.org and done. We need to build our player for each platform.

Thanks for the EasyRPG team to make something super buildable from scratch (they even make a dedicated repo for building all the dependencies in bulk at https://github.com/EasyRPG/buildscripts ), this was not very complicated task (unless you are new in that world, such as me at 2018). Of course this building process for each OS had it’s issues there and there. Let’s manage some of those:

Windows

I will skip this part. I already told you about mistakenly trying to build Player using MinGW instead MSVC. The happy part of it is that we were able to provide a nice 32bit custom player. (Yes, 32 bit for that guy that still had issues running a 64 bit OS… which should be 0.00001% of our population playing the game).

MacOS

For building a player for MacOS is easy: You need a Mac. And have money for one. I don’t have either of those XD. Luckily the Hackintosh community and efforts to run tweaked versions of OS running on PCs is quite big out there. At least enough to find hacks for VMWare to run MacOS EFI firmware that allows me to install a stripped-down version of MacOS High Sierra at that time, with enough stuff to run a terminal, and a Xcode.

But wait there… You also need to install all build tools (which comes with git, make, clang, etc, etc)… Or go by the easy way and Install Xcode. I remembered spent 3 days trying to find the correct version to download (cuz the official way via Apple Store did not worked). Once I was able to find it, I did not had much hassle to build a functional version:

(It even have a custom About screen, yay!)

For those people who has built software for MacOS they know I missed some steps there (that Steam at that era did not cared about) such as Notarization and signing. At that time these were not required, but surely that’s not the scenario today. Most probably you will have to tackle lots of hefty warnings and alerts of unsecure software to make this run nowadays. Not sure if signing for publishing in Steam is required, but at least for notarizing you will need a Apple Developer ID. And yeah, that cost money! Duck you Apple!

(Thanks to Ali Rastegar for lending me a hand here; he already integrated some MKXP games with Steamshim in MacOS ).

GNU/Linux

That should be easy-as-a-pie building something here, right? Of course it is! I just grabbed an Ubuntu ISO, installed it somewhere, installed buildtools, built everything (remember to build statically) and everything seems nice at the end of the day! 

(A Linux binary without 350gr of logs is not a Linux binary). 

But the nightmare started when I wanted to run the game in another PC with another Linux OS. The thing was badly dead. ldd tool told me that my shiny executable was strongly tied to a very specific version of libc I used from my version of Ubuntu, so it can be run only in a small number of places. And you know, there is no such thing called “standard” in GNU/Linux where I can compile this thing to run it in a bigger stake of OSes (put Debian, RedHat, SuSe, arch and others in a cage and grab a popcorn to see how it ends… if it ends XD).

 Luckily, the solution from that comes from Steam Itself: They offered a thing called Steam Runtime   that offers a pretty-old Ubuntu 12 based runtime to build stuff for Steam and attempt to make it compatible with a broader steak of OSes. Today that environment is Docker based, but before that it was some sort of strange chroot stuff. Struggled a bit to make things run to build a functional player, but after dealing for a while I was able to accomplish my objective: Making a functional custom EasyRPG Player able to run in whatever platform able to run Steam. Yay! 

Final touches before publishing. 

I have the things ready to be published! I must ensure some random stuff:

I need to accomplish with GNU GPLv3: Okay, put license files on the game data files, plus publishing the custom player source code at https://github.com/elsemieni/easyrpg-player-netherware (warning: very outdated).

Optimized audio formats: If EasyRPG offered OPUS, then let’s use OPUS! (Well, after dealing with an issue that made player crash when playing opus files on non AVX enabled processors).

Loooooong pre-launch QA: Outside of the scope of this post, but it was looong and tedious!

Add proper controller support: At that time, EasyRPG Player 5.4 had not a pretty well controller mapping (even worst if applied to Desolate City). So I brought an Xbox One controller from a buddy in my faculty for around $20 (from my own wallet) to remapping and testing! Worth it!

(CesarRP, you owe me $20… nah, just kidding).

Easter eggs: shhhhh. Did you guys played Moulin la Folie? It’s hidden somewhere inside Desolate City :D

(No, I will not tell you how to unlock this game)

Aftermath... wait, there are math in the after?

I guess CesarRP have better words than me for that chapter; but at least I can say the game was successfully launched in the markets, and you can play it too here. 

Of course the game industry never stops, and after that Super Crystal Hunter  , another creation from Netherware came.

Here the same basis than all before was applied (with the sole difference that Player 0.6 was used and the customizations were made a bit cleaner at https://github.com/elsemieni/easyrpg-player-ne-sch ). Just mention some extra stuff customization made there:

FluidSynth: Needed as MIDI sounds different in non-Windows OS. At that time, FluidSynth backend was not ready for Player, so made some hacks from a experimental branch (from Ghabry I guess) and dealt with the issues and workarounds. So we were able at that time to use a gm.dls clone soundfont to make it sound similar in all Oses. (An interesting thing is that the trailer music was the main theme with a custom soundfont working on Player’s Fluidsynth).

Rumble: Absolutely not needed, but wanted to play with it because why not xDDD. Created dummy special sounds that when you play it on Player it will rumble the controller, depending of the Vol/Pan/Speed parameters. So, yeah. SuperCrystalHunter has rumble support (in a game which is completely useless but … meh, don’t ruin the fun).  

Conclusions, issues and the future…

Of course this approach to implementing Steamworks SDK (or any other backend) to EasyRPG Player games have a particular issue: Each time you want to have a up-to-date player, you should update the codebase of your custom player (with the risk of breaking it) and recompile for all your platforms. Had to do that more than once and it’s tedious.

Probably in a very professional scenario you want to stick to a particular version of Player instead of always having the super latest version (that’s a very common practice in popular engine such as Unity or Unreal Engine). And that is what I decided for Desolate City (Player 0.5.4) and Super Crystal Hunter (Player 0.6). CesarRP did not like much the idea at that time as far as I remembered, and requested me sometimes to update the codebase.

One way to avoid this could be implementing something in Player such as the possibility to call external DLL/SO/dylib stuff (such as the Win32API module in RPG Maker XP/VX(a)). However I believe this will not be possible due 2 reasons: Very OS dependent, and very risky in terms of security. Then, I guess the alternative to customizing the base code is still the right option to do so. Also I think is the same reason of why ExecProgram feature of KeyPatch is not implemented. (Let’s not put DynRPG here, because this is other sort of issue… as that is special crafted compiled code made to hack an specific version of RPG_RT itself).

And of course, I already mentioned the GPLv3 vs MIT issue at the middle of this post (reminder of this discussion here) that could allow implement not just Steamworks API more cleaner, but also more heavy SDK such as game consoles! But yeah, at the cost of sacrificing Freedom (some people strongly believe in that part as core pillars of EasyRPG: A free-as-in-freedom RPG Maker). Anyways, the debate is still hot sometimes there. 

I want to add Steamworks into my EasyRPG game. What should I do? 

Probably plenty of people are here specifically for this section, so let me dedicate a few words for it.

I will start saying is not something pretty easy as "install this module/script/plugin into your game, configure some things and that's it". You will need to get your hands a bit dirty and tweak a bit the Player source code for accomplish that, so expect having some knowledge with C/C++ and CMake.  

Maybe this is not a technical how-to, but here are some clues about how to proceed:

• Get Steamshim, play with the included examples (uses Postal, it's free on Steam) to get it working and make yourself comfortable with it.
• Add Steamshim to EasyRPG Player source code, add a hook somewhere to call archivements from your game (there is not a guideline for putting specifically in any place... I put it in at "End Event Processing" event command, but you are free to put whathever you want).
• Build that custom player in your platform of your choice. I put some memos upper in this post about how I went at Windows/MacOS/Linux. 
• Do not forget you are required to publish also your modified Player source code, as it's a requirement of GPLv3 license. 
• Order a pizza. This is mandatory (?)

Oh, and this is the end of the post! Had to make it in two whole days (or nights to be more specific). Now I can rest at least 2% more peacefully, as at least I make this testament about how all of this was made.

As always you can always ask me about this stuff in a channel I can be available of course!

Now, I will go to sleep a bit, 

byeee!

So... what happened?

Hey hi there! 

(At least 404 horse is someone still watching the blog - a photo from mine not so long ago)

You don't expected myself writing there after... around 12 years after. In fact, let me clean all the spiders around there (well, not really, it's a web page, but at least I cleaned all those broken imageshack (RIP) URL's and broken stuff cof cof*). 

 Let's assume like nothing happened and let's do a little life resume, such as the good old times.

I said at that era that good things were coming in a new, shiny blog hosted at my new purchased domain at the time - elsemieni.net, hosted in a crappy hosting company around somewhere (I would love to talk about that in a future post)... Well, that didn't worked obviously, but while that happened, something called life happened.

You know, the typical cliche stuff, studies, studies, more studies (if you ever doubt), got my professional title, learning a lot of weird stuff, seeing things at internet fall, while others arising (I'm talking to you Forums, Instagram and Discord), getting old, start having my hair grow, cut, grow again, cut again (definitively keeping it short), the auge of EasyRPG Player, helping friends we knew in game making groups publishing games into Steam (anothern topic I will love to talk in another post, with permission of Netherware entertainment ), had a good time making some homebrew scene ROMs (this and this), made a lot of stuff with Phaser and Raylib , discovered the demoscene (and figuring I'm too lamer for being there still) and still making weird stuff with RPG Maker, (which includes some games too )  falling into that thing called love twice (then lots of character development canon-histories), life decisions, a pandemic, wishes of flew up the continental country (and failed again haha), ending working in a dream-job in the middle of nowhere monitoring 66 antennas in a some sort of astronomical complex... and a loooooooooong list of etc. 

(An example of weird experiments making last years with RPG Maker... yeah, still on that world)

You will see, life happened. And that was intense for saying at least.

But something is missing here. Something I missed for a while in all those years, specially with the fallout of forums systems, at the fun being at RPG Maker communities... a place for sharing nerdy things. Yeah, I know, I'm getting old and maybe I'm not into those tiktok stuff or maybe not knowing how to share bulky stuff in those fast chat systems such as being a Youtuber, or joining Discord communities. (tried, failed, but found some nice communities with ever nice people where I still I'm into).

Okay, I guess I need to re-start publishing my nerdy stuff here at Blogger. I know, it's an outdated thing but hey! It works! (Well, still... hope Google will not wipe it in the next years in middle-terms). I linked to my homepage and domain ( https://elsemieni.net ). Probably almost nobody reads those posts but... hey! It's a least a place where I can dump my stuff without staying in volatile systems (I'm talking to you Wordpress!) or volatile social platforms. 

I want to talk about weird stuff I make sometimes, RPG Maker mainly, projects, music, etc. Knowing that probably making community here will be a bit difficult, at least it's a safe place where put things :) Anyways, also I guess you already noticed, after all those years, I think I'm able to write in english (a bit crappy sometimes, but it works! And no, I will resist myself to use gramarly or similar stuff like that), so from here It will be my main blog language. Hope you don't get mad about :D (And if you are, well, it's not my ducking problem).

I won't warrantee you that I publish stuff in a regular basis (even there is the chance that I will never touch the blog again once in 12 years), but I will try to when I do stuff at the life.

So, nothing more to say, welcome again to EN.I's blog! Or EN.I's tabern (Or elsemieni's blog... but I don't want to be confused again with ElSemi from the emulation retro scene which is someone else much more talented than me hehe...)

PS1. Post'datas abbreviature sounds like PlayStation, huh?

PS2. Did not wanted fancy themes now for the blog, but also did not wanted something too liberalish. Just wanted something that works. 

PS3. As you can see, lots of good old RPG Maker communities just dissapreared over the time. If you want to hang out with good old friends, join us to Comunidad RPG Maker.es community at Discord https://discord.gg/3fkteJJ (Or Matrix.org if you are electro-anarchic https://matrix.to/#/#rpgmaker.es:matrix.org #rpgmaker.es:matrix.org )