Runtime Source Dependencies and Linux Runtime Include bug fix

The latest code on Github includes a new feature - Runtime Source Dependencies - along with a bugfix for Linux builds which resolves an issue with Runtime Includes.

A runtime source dependency is a source file which you need to have compiled when a given header is included by a runtime modifiable source file. Whilst having the source in a library and using the Runtime Link Library feature solves this problem to an extent, sometimes you don't want to create a whole library and thus the ability to compile in dependencies is a really useful feature. Using this feature simply requires the following lines in a header:

#include "RuntimeSourceDependency.h"
RUNTIME_COMPILER_SOURCEDEPENENCY;

If the header is called SomeFeature.h then the source file SomeFeature.cpp is compiled when any runtime modifiable code is changed which includes this header. Using the same filename as the header is required at the moment, in part due to issues with getting full paths from builds on Linux with GCC.

This brings us to the bug fix on Linux - although we'd implemented a system to get the full path for runtime modified source files from GCC, this hadn't been implemented for runtime includes and making the above changes caught this bug, so it's now been fixed. GCC only embeds the path passed in for __FILE__, which can be a relative path from the compile location, so we have to embed the compile path using the pre-processor define COMPILE_PATH="$(PWD)/"; Note that this isn't required if your build system uses full paths, which the cmake builds do, as does the internal GCC runtime compiler.

Note that I'm experiencing an odd issue on my cmake builds on Linux with makefiles - the Eclipse generated files work well however the makefile ones compile but the SimpleTest program fails to create a glfw window.

Added cmake files

Thanks to Anders Buus we now have a set of cmake files for the project.

Linux port of SimpleTest game example

The Pulse demo from project SimpleTest running on Ubuntu Linux compiling changes at runtime.

The SimpleTest example project, otherwise known as the Pulse game demo, now works on Linux with the Eclipse IDE. There's a known bug in that the app shut down appears to crash, which I'll investigate and fix as soon as possible. Grab the latest from Github to see it in action.

[Edit 2013/03/12:] The crash on exit bug for the SimpleTest example has now been fixed.

Linux port with Eclipse progress update

Yesterday I started porting RCC++ to Linux, using the Eclipse IDE as this seemed the favourite option, or at least that's what's been specifically asked for. It's been a while since I've done any serious Linux/Unix coding, and that was in the days before decent IDEs using vi and simple makefiles, so it took a fair few hours to get done. There's now a working version of the runtime compiler and ConsoleExample in my personal fork of RCC++, which I'll pull onto the main fork after some further testing.

[EDIT 2013/11/03]: The code (with a few fixes) has now been pulled into the main fork at Github.

The base RCC++ libaries and the ConsoleExample have no dependencies, so the most difficult problem was getting __FILE__ to reliably generate full paths, since Eclipse builds each project from the path of the output. This required embedding the compile path using COMPILE_PATH=\"$(PWD)/\" as a preprocessor define option (-DCOMPILE_PATH=\"$(PWD)/\" on the gcc command line), which uncovered some bugs in my FileSystemUtils header. Note that the executable needs to be build with the flag -export-dynamic in order to allow symbol resolution by name.

I've not yet ported the rather complex SimpleTest (which needs renaming) due to the dependencies, but this shouldn't stop anyone from using the base libs. Additionally, I'm a little at a loss as yet as to the best way to distribute multiple eclipse projects which are part of a hierarchy. For now you'll need to manually add them to your workspace, which eclipse makes easy by defaulting project files to being hidden. I feel that I've not yet spent enough time with the IDE to complain, but it seems like a good part of a programmers life is fighting the build systems rather than using the language.

Boost removal, improved library support and VS 2012 builds

We've developed a cut-down implementation of the boost::filesystem features we required in our FileSystemUtils.h header, allowing us to remove our dependency on Boost. As ever this is permissively licensed so feel free to use that file if it meets your needs outside of RCC++, and extend / modify as you need.

Additionally, we've added the capability to link libraries at runtime, in both Windows and Mac OS X builds. A simple example for how to use this with the glewfw library is to create a header which you include instead of the glewfw.h header as follows:

#pragma once

#include <GL/glewfw.h>
#include "RuntimeLinkLibrary.h"

#ifdef _WIN32
RUNTIME_COMPILER_LINKLIBRARY( "glewfw.lib");
RUNTIME_COMPILER_LINKLIBRARY( "opengl32.lib");
#endif
#ifdef __APPLE__
    RUNTIME_COMPILER_LINKLIBRARY( "-framework glewfw");
    RUNTIME_COMPILER_LINKLIBRARY( "-framework OpenGL");
#endif

When included by runtime compiled code, the library will be linked during compilation as and when required. Note that for OS X, we need to specify frameworks with "-framework [name]" and "-l[name]" where [name] is the framework or library (see ld documentation on Apple's developer site). You can link to both static and dynamic libraries this way, but the use of dynamic libraries should be preferred. With dynamic libraries only one instance will be loaded by the executable, and resources (such as graphics data) can be preserved without serialization between runtime compiles.

Being able to use libraries is a significant improvement in functionality. For example, core rendering functionality can now be runtime compiled.

Furthermore, we've fixed up the Visual Studio property sheets so the solution will now build with Visual Studio 2012 (including the express version), though you'll need to upgrade the solution so that it uses the default VS 2012 toolset (called v110) if you want or need to. Visual Studio should prompt you to upgrade the solution when opened.

There's quite a lot more, For full list check out the commits..

Mac OS X and Simultaneous Multi Platform Runtime Compiling

With the help of the lightweight cross platform GL windowing library GLFW and the Simple File Watcher, we've added Mac OS X support to Runtime Compiled C++. As an added benefit, you can now run multiple platforms from a shared directory, using one of them to code on the fly. Turn around times on Mac OS X with XCode 4 and Clang++ are excellent, better than windows on a similar system.

Mac OS X 10.8 with XCode 4 running the SimpleTest example natively top right along with a Windows 7 Parallels Desktop Coherence mode version of the example running bottom right. The apps in each OS have just compiled changes made from XCode. Turn-around times are significantly faster when not running the VM.

For the moment you'll need OS X 10.8 and XCode 4 to run the code on the Mac (we've not tested with OS X 10.7 but it might work). If you change the debugger to GDB the exception handling mechanism doesn't work as smoothly as with the default LLDB debugger, since GDB doesn't pass the exception signal on to the app.

We'll have more information on the port, the changes and our plans for the future soon.

Please do give us your feedback on the OS X version, as we've had limited ability to test this on other systems as yet.

Building a Console

Building a Console from RuntimeCompiledC++ on Vimeo.

Many game engines include a command console that allows a user to issue text commands to adjust settings, get debugging information, change the state of the world and so on. The feature opens up a great deal of potential for rapid development.

However, the implementation will usually be based on a scripting language of some kind: either giving access to one already integrated or using simple custom syntax. Both of these represent significant barriers and maintenance costs. To allow you to access all the features of your engine a scripting language must be pervasively integrated, which is a lot of work - and if you've been reading this blog, you know we see a lot of disadvantages in this approach. A simple custom scripting language will also take work and will always be much less powerful.

RCC++ offers a new option: allow the user to give commands directly in C++.

We can do this by taking the text the user has entered and dropping it into a simple .cpp file, forming the content of a standard execute() method. We can then compile this file at runtime and immediately execute it, just once, as a "snippet" of disposable code. By passing in virtual interfaces to our major subsystems we can have access to our existing functionality with very little work. In essence, anything you could do in the C++ code, you can now do on the command console.

The same crash protection discussed before allows us to catch basic crash errors, so we can use it with confidence. We can also go further: if we restrict the symbols and methods we expose, we can make it safer to use; if we write some nicer wrapper methods we can make it easier to use. This opens the way for a console friendly enough for non-programmers.

We've got some other ideas for applications of code "snippets" and making them easier for designers to use - you can expect this topic to return in future posts.