Fatal memory overflow in specific seeds in C++

[Desperationis]

I don't know how to explain this. I've literally debugged this for days, even rewrote my multi-file project into a one-file one, and I am still not sure what causes it. I am on Linux, and program in the C++ v1.1.x kit.

Basically, there are certain seeds that cause my bot to crash in one gigantic memory overflow, which is always near turn 340-360. The error always looks like this, though memory access out of bounds can also happen:

[Agent 0 Log] Cannot enlarge memory arrays to size 16781312 bytes (OOM). Either (1) compile with  -s INITIAL_MEMORY=X  with X higher than the current value 16777216, (2) compile with  -s ALLOW_MEMORY_GROWTH=1  which allows increasing the size at runtime, or (3) if you want malloc to return NULL (0) instead of this abort, compile with  -s ABORTING_MALLOC=0 
[Agent 0 Log] exception thrown: RuntimeError: abort(Cannot enlarge memory arrays to size 16781312 bytes (OOM). Either (1) compile with  -s INITIAL_MEMORY=X  with X higher than the current value 16777216, (2) compile with  -s ALLOW_MEMORY_GROWTH=1  which allows increasing the size at runtime, or (3) if you want malloc to return NULL (0) instead of this abort, compile with  -s ABORTING_MALLOC=0 ) at Error
[Agent 0 Log]     at jsStackTrace (/home/diego/Desktop/LuxBot/main.js:1807:19)
[Agent 0 Log]     at stackTrace (/home/diego/Desktop/LuxBot/main.js:1824:16)
[Agent 0 Log]     at abort (/home/diego/Desktop/LuxBot/main.js:1543:44)
[Agent 0 Log]     at abortOnCannotGrowMemory (/home/diego/Desktop/LuxBot/main.js:1931:7)
[Agent 0 Log]     at _emscripten_resize_heap (/home/diego/Desktop/LuxBot/main.js:1936:7)
[Agent 0 Log]     at <anonymous>:wasm-function[3226]:0x60e6e
[Agent 0 Log]     at malloc (<anonymous>:wasm-function[3220]:0x5edc8)
[Agent 0 Log]     at <anonymous>:wasm-function[3002]:0x5acdc
[Agent 0 Log]     at <anonymous>:wasm-function[102]:0x6b50
[Agent 0 Log]     at <anonymous>:wasm-function[57]:0x41ba,RuntimeError: abort(Cannot enlarge memory arrays to size 16781312 bytes (OOM). Either (1) compile with  -s INITIAL_MEMORY=X  with X higher than the current value 16777216, (2) compile with  -s ALLOW_MEMORY_GROWTH=1  which allows increasing the size at runtime, or (3) if you want malloc to return NULL (0) instead of this abort, compile with  -s ABORTING_MALLOC=0 ) at Error

Keep in mind that this only happens rarely, like 1 in 50 seeds.

Here is a one file version of my bot using the cpp kit in the v1.1.x branch:

#include "lux/kit.hpp"
#include "lux/define.cpp"
#include <string.h>
#include <vector>
#include <set>
#include <stdio.h>
#include <map>

using namespace std;
using namespace lux;
int main()
{
	kit::Agent gameState;
	gameState.initialize();

	while (true)
	{
		GameMap& gameMap = gameState.map;
		Player& player = gameState.players[gameState.id];
		vector<string> actions;
		gameState.update();

		vector<Cell*> resourceTiles;
		for (int y = 0; y < gameMap.height; y++)
		{
			for (int x = 0; x < gameMap.width; x++)
			{
				Cell *cell = gameMap.getCell(x, y);
				if (cell->hasResource())
				{
					resourceTiles.push_back(cell);
				}
			}
		}

		for (int i = 0; i < player.units.size(); i++)
		{
			Unit& unit = player.units[i];
			if(unit.isWorker() && unit.canAct()) {
				
				if(unit.getCargoSpaceLeft() != 0) {
					ResourceType resourceToFind = ResourceType::wood;

					if(player.researchedCoal()) resourceToFind = ResourceType::coal;
					if(player.researchedUranium()) resourceToFind = ResourceType::uranium;

					Cell* closestResourceTile = nullptr;
					float closestDist = 999999;

					for(auto it = resourceTiles.begin(); it != resourceTiles.end(); it++) {
						Cell* cell = *it;
						float dist = cell->pos.distanceTo(unit.pos);
						if(cell->resource.type == resourceToFind && dist < closestDist) {
							closestDist = dist;
							closestResourceTile = cell;
						}
					}

					DIRECTIONS dir = unit.pos.directionTo(closestResourceTile->pos);
					actions.push_back(unit.move(dir));
				}
				else {
					CityTile* closestCityTile = nullptr;
					float closestDist = 999999;

					for(auto city : player.cities) {
						for(auto cityTile : city.second->citytiles) {
							float dist = cityTile->pos.distanceTo(unit.pos);

							if(dist < closestDist) {
								closestCityTile = cityTile; closestDist = dist;
							}
						}
					}
					DIRECTIONS dir = unit.pos.directionTo(closestCityTile->pos);
					actions.push_back(unit.move(dir));
				}
			}
		}

		for (int i = 0; i < actions.size(); i++)
		{
			if (i != 0)
				cout << ",";
			cout << actions[i];
		}
		cout << endl;
		// end turn
		gameState.end_turn();
	}

	return 0;
}

To get the same error, you must crosscompile main.cpp into a JS file then run the main.js it against itself in seed 556459304 using the v1.1.x cpp kit.

Here are noteworthy findings:

• You don't get this error if you run the cpp file directly. If you manage to compile cpp as discussed in Can't directly run cpp files in lux-ai-2021 (ETXTBSY error) #71, you never get this type of error.
• The error seems to indicate the program only has like 16mb of memory.
• Replay doesn't show anything noteworthy. I spent hours analyzing the last movements of the program before it crashed and nothing seemed wrong.
• Happens rarely, like 1 in 50 odds
• The program always, I mean ALWAYS, crashes when you call kit::Agent.update(). Debugging via cerr showed that the entire program run fine with no issues whatsoever right up until kit::Agent.update().

[StoneT2000]

Looks like adding -s INITIAL_MEMORY=X to the compile command is sufficient. Had the error you mentioned earlier but it was gone after adding this.

We will make this a default in the standard c++ kit. Let me know if setting X to something like 128mb works, so do

-s INITIAL_MEMORY=134217728

Full compile command is

emcc -s FORCE_FILESYSTEM=1 -s INITIAL_MEMORY=134217728 --pre-js internals/init_fs.js main.cpp -o main.js

[Desperationis]

@StoneT2000 That did the trick! Thanks :)

[StoneT2000]

fixed in #51