malmo

Addresses various async io issues on MacOS. For use with the MarLo crowdai competion.

With new Python only implementation of Open AI gym environment.

For Malmo to be used widely for both exploring and teaching reinforcement learning in a games environment, it needs to be simple to build and install on multiple operating systems. This release addresses some of the cross platform build issues by removing the C++ XSD code generation and using a modern version of CMake (without custom extensions). This unfortunately entails dropping direct support for Lua and Mono, focusing the project on Python (version 3), while still providing direct support for C++; C# (on Windows and hopefully supporting .NET Core soon) and Java. Other languages that can link to C/C++ (Lua included) can easily be added using Swig (http://www.swig.org/), JVM or other cross language interop mechanism.

We can now provide a pip3 installable (native) Python module (“pip3 install malmo”) for Windows, MacOS and Linux. There are some remaining strict dependencies on Python 3, Boost and Java 8 that still need to be pre-installed but we hope that being able to install Malmo from the network will make it significantly easier to start using Malmo.

Other changes in this release address the set-up and reliability of communications between the Malmo agent and the Minecraft Mod.

Changes:

It's been a while since we had a release, and a lot of work has been going on, so we've skipped straight to 0.34.0.
The main focus has been on Malmo's usefulness as a tool for gathering data, particularly in imitation learning scenarios. A secondary focus has been on stability.

• New: Can now record video streams as individual frames rather than video - eg my_mission_record.recordBitmaps(MalmoPython.FrameType.DEPTH_MAP).
  This allows for greater fidelity when recovering frame-action pairs, bypassing the data loss that comes from encoding to mpeg and decoding back to frames.
  8bpp luminance and 24bpp colour frames are saved as pgms and ppms respectively. (Saving as pngs is too slow to keep up with the frame rate.) 32bpp depthmaps are saved directly as numpy arrays. (See depth_map_parser.py in the new scripts/tools folder for code which can convert these into images.)
• New: "Human" action space and observations, for tighter human-in-the-loop scenarios. This allows us to record mouse and key events directly, through the Minecraft window, rather than requiring the separate human action component. Add the ObservationFromHuman handler to your mission xml, and take control of Minecraft (by hitting the Enter key) - Malmo will receive mouse deltas and keyboard events. There is a matching HumanLevelCommands handler which can replay the recorded events.
• New: Option to restart Minecraft from the agent. agentHost.killClient(clientinfo) will cause Malmo to kill that particular Minecraft client and start up a new one - provided that launchClient.bat/sh was run with the -replaceable flag. This is useful for long-running experiments - we've found that Minecraft becomes unresponsive over time, and periodically restarting it can be helpful.
• New: ObservationFromSystem observation handler to return stats on Minecraft's health (eg render/server/client ticks per second) - use in combination with the restart option to keep Minecraft healthy.

Other additions:

• New: Titles and subtitles are now returned as part of the chat observation.
• New: DrawSign allows signposts to be drawn with specific text.
• New: 'includeNBT' flag for ObservationFromRay, to return JSON NBTTagCompound for tile entities.
• New samples:
  • 'decision_tree_test.py' - demonstrates DrawSign and the includeNBT flag.
  • 'braitenberg_simulation.py' to test luminance video producer (and for fun)
  • 'mouse_steering_test.py' to test human action space

Fixes:

Many fixes in the area of mission recording - this should now be much more stable.
In addition, the python samples have been tweaked so that they can be run headless as part of the integration test suite.

Extra scripts:

From time to time we find ourselves creating python scripts to carry out some data-processing task on Malmo recorded data, or Malmo log files, etc. These now have a home in scripts/tools, on the grounds that they might possibly provide useful material for carrying out other tasks in the future.

What are all these Dockerfiles?

In a bid to simplify the Malmo build process, we now build the release binaries using docker containers. The dockerfiles in scripts/docker set up build environments, with all necessary dependencies installed. This should make releasing new versions of Malmo significantly easier. They can also serve as a form of documentation - consider them to be a more detailed and up-to-date (but slightly less readable) version of the doc/build_linux.md file.
You can also use them to create your own build environment, should you want an easy way to grab a build of the latest code from master. Put the kettle on while you wait for the docker container to build though.

Support

We are beginning to drop support for some of the older Linux distributions. In particular:

• We are no longer testing against Ubuntu 14.04. The version of mesa which ships with 14.04 is very outdated, and contains a bug which makes it impossible to run the integrated tests headless. We've provided 14.04 binaries, but they are untested.
• Debian 7 is included for now, but will be dropped soon. Note that the version of Python 3 which comes with Debian 7 is too old to work with Malmo's Python samples and tests. (Malmo uses future to make the Python code cross-compatible with Python 2 and 3, and this requires Python 3.3 as a minimum.) We've included the Debian 7 Python 3 build for completeness, but it's untested.

Changes:

HEADLINE CHANGE: New Video Producers!!!

• New: 8bpp luminance producer. Why waste time sending 24bpp RGB video frames across the ether, and then waste the agent's time and effort converting them to 8bpp greyscale? Just send the greyscale image in the first place.
• New: 32bpp depthmap producer. The old 8bpp depthmap (sent on request in the alpha channel of the standard video producer) is retained for backwards compatibility, but you can now request a more precise depthmap, which has been scaled for you to "real" values - eg each pixel represents the distance that pixel is from the agent's eyes in block units.
• New: 24bpp "colour map" - essentially labelled data for use alongside the standard video frame. Each pixel is coloured according to the block, or entity, that produced it. At the moment there is no control over the block colours - they are chosen for you - but in a future release we will add the ability to specify this (eg all gold blocks are currently coloured RGB(232,23,70), but in the future you could request that gold blocks are (255,0,0), and all other blocks are (0,0,0), etc.) There is control over entity colour, so you could, for example, colour all "friendly" entities pink, and all hostile entities green. You can also set the sky colour.

Other changes:

• New: ability to draw chests / other containers, and determine what should go in them.
• New: inventory observations now include inventory of any containers under the agent's gaze.
• New: inventory commands can now swap items between the player's and container's inventories.
• New: ender chest contents can be specified for each agent.

Breaking changes:

Python 3 support!

All our sample/test python code has been "futurised" to work with both Python 2 and Python 3. This shouldn't be a breaking change, but there may now be an extra dependency to install - try pip install future if the samples don't seem to be working.

Changes:

WE'VE UPGRADED TO A NEWER MINECRAFT!!!!

It was time - while we've been stuck with Minecraft 1.8, four new versions have come out...
We've updated to almost the bleeding edge - Minecraft 1.11.
This means:

• New: Mobs - shulkers, vexes, illagers, polar bears and llamas and many more - check out the mob_zoo.py sample for a complete list.
• New: Blocks and Items - check out the changes in Types.xsd

Breaking changes:

There have been significant changes to Minecraft, and Forge (which we use for modding), so there may well be some new quirks we've not yet spotted in our testing. The changes we know about are:

• AllowSpawning/AllowedMobs now affects mob_spawners
• "Minecart" entity has been renamed to "MinecartRideable"
• There is a known issue whereby agents with no explicit starting position can sometimes "sink" into the ground at mission start.

Other new features:

• New: Added support for drawing tuned note blocks - see the "note_block_test.py" sample

Changes:

The bulk of the work for this release has been centered around adding logging code to both the Mod and platform, which should significantly help us to track down and fix any future issues with Malmo's socket code, etc.

You can switch the Mod logging on by going to the Mod options (in the Minecraft GUI) - be warned, if you turn the detail up too high this will produce a LOT of logging...
The logs are stored in the logs subdir of the Minecraft folder, and are named after the date/time when that Minecraft instance was run.

On the platform side, switch it on by doing something like this:

MalmoPython.setLogging("where_I_want_it.log", MalmoPython.LoggingSeverityLevel.LOG_ALL)

Again, this will generate a lot of noise unless you opt for something like LOG_WARNINGS, etc.

The two log formats are the same - they start with a timestamp formatted the same way, followed by an "M" for Mod or a "P" for platform, so in theory the two files could be merged and sorted together, which should make forensic debugging a bit easier.

Users can append a line to the platform log by calling, eg, MalmoPython.appendToLog(MalmoPython.LoggingSeverityLevel.LOG_INFO, "Here's some important info...")

This is just intended to make debugging easier - it's not intended as a general purpose log for the user's code. It can be used to add signposts to the log - eg "INFO: started episode #242" which should make searching for problems easier.

Other new features:

• New: ObservationFromRay now returns distance.
• New: ObservationFromFullStats now returns DamageDealt
• New: startMission now provides more error details in exceptions; multi-agent samples hardened by using this.
• New: ObservationFromNearbyEntities now returns entity life.
• New: RewardForDamagingEntity plus sample. (#537)
  Assorted bug fixes

Breaking changes:

Hopefully nothing...

Changes:

New features:

• New: yaw and pitch added to nearby entity observations.
• New: reward handler and quit producer for mob catching. (#440)
• Assorted bug fixes

Breaking changes:

Hopefully nothing...

Changes:

New features:

• Turn scheduler - for multi-agent scenarios where each agent must waits its turn. (#441)
• Moving target decorator - for scenarios where agents need to chase things - this provides a simpler, more easily controlled thing for them to chase. (#458)
• Scripts to help with building/installing Malmo - these are currently works in progress - use them at your own peril! The linux script currently only supports Debian and Ubuntu. Even if they don't work perfectly, they may serve to speed up the process of installing or building.

Breaking changes:

Hopefully none!

Which version?

Unless you want to use the Arcade Learning Environment, avoid the _withALE builds.

Changes:

New features:

• Multi-agent scenario reworked - now possible to run multi-agents in an offline LAN setting, so no authentication required.
• Version checking - schemas, mod and platform must all have the same version number.
• Time-based rewards. (#126)
• Discrete jumpuse action (great for nerd poling). (#400)
• Team rewards - agents can now work in teams, and share rewards with each other. (#279)
• Discrete strafe and jump actions, autojump and autofall. (#352, #321)
• Simple animation decorator - have fun animating blocks... (#389)
• Boost can now be linked statically, and these release binaries have done so. (#392)

Breaking changes:

Hopefully nothing... but the work on version checking is fairly drastic - if versions don't match, fatal errors will be thrown. This should allow everyone to catch potentially confusing version mismatch errors straight away; keep an eye out for error messages!

Linking statically with boost is also a fairly major change to the releases. It should put an end to errors like #427, #395, #179 etc.etc. If building from source, boost can be linked dynamically as before.

Which version?

Unless you want to use the Arcade Learning Environment, avoid the _withALE builds.
The Redhat build was made with Fedora 23 but may work with other Redhat flavors.
The Ubuntu 16 build was built (statically) against boost 1.62.

Changes:

It's a while since we've built a release, so quite a lot has gone into 0.18...

New features:

• Build battles - added RewardForStructureCopying and BuildBattleDecorator. (#337)
• WorldTime and TotalTime added to ObservationFromFullStats. (#262)
• Agent can now quit by using MissionQuitCommands. (#170)
• Python sample to test initial chunk loading. (#338)
• Added startAtWithPitchAndYaw to API. (#295)
• Greater control over drawing of rails and stairs. (#323)
• Entities can now be placed by DrawingDecorator. (#322)
• ObservationFromGrid has new optional attribute absoluteCoords for fixed-location grids. (#293)

Fixes:

• Three second grace period on spawning removed. (#88)
• AbsoluteMovementCommands can now teleport outside the current chunk. (#353)
• FileWorldGenerator load issue + file_test.py to test fix. (#342)
• Weather is periodically reset to mission requirements. (#264)
• ContinuousMovementCommands now allow the setting of yaw/pitch by Discrete or Absolute movement commands. (#255)
• Continuous-mode use/attack weren't correctly triggering rewards for discarding/collecting items. (#303)
• Discrete-mode use/attack weren't correctly triggering rewards for discarding/collecting items. (#297)

Breaking changes:

Most of these are subtle changes which are unlikely to affect anyone, but they do carry the potential to change results / break existing code:

• Fix: The centre of the grid observation was being rounded to the nearest int in the y-axis. (#370)

  This will change your results for cases where the agent is in between blocks in the y-axis, eg if they are jumping / falling / climbing stairs. The grid observation will now be centred on floor(y) rather than floor(y+0.5)

• Fix: temp folder and "mission_recordings" now removed from path of files in recordings. (#298)

  This means the folder structure of your mission recording files has changed - if you've written any automated tools for parsing recordings, you may need to tweak them.

• New: AbsoluteMovementCommands tpx, tpy, tpz now applied immediately.

  Previously you could combine these commands by sending them immediately after each other - eg to teleport from the current location to (4,200,12) you could send:

  tpx 4
  tpy 200
  tpz 12
  

  and (most of the time) all three actions would be applied together at the next tick. In fact, in previous versions there was no other way to do this, since we hadn't provided a tp x y z command.
  This latest change, however, means that each individual action will be applied consecutively, so if the agent starts at, say, (16,32,64), they will now visit the intermediate points (4,32,64) and (4,200,64) before arriving at (4,200,12).

• Fix: Extra block properties returned by ObservationFromRay are now prefixed with "prop_" to prevent collisions. (#355)

• New: Mission worlds are now deleted after use unless destroyAfterUse is false. (#76)

  The default is for destroyAfterUse to be true - so, by default, starting with this version, any Minecraft saved worlds that get created to run the mission will eventually be deleted again after use. (Previous saved worlds won't be touched, and your own worlds should be safe.) If you rely on your saved worlds hanging around, make sure you explicitly set this flag in your XML.

Which version?

Unless you want to use the Arcade Learning Environment, avoid the _withALE builds.
The Redhat build was made with Fedora 23 but may work with other Redhat flavors.
The Ubuntu 16 build was built against boost 1.61; if you still have boost 1.58 installed then use the Ubuntu 15 build.

Changes:

Fix: Discrete use and attack now affect the inventory. (#247)
Fix: Rewards would occasionally get doubled. (#275)
Fix: Stopped mobs spawing illegally. (#280)
New: Added has_mission_begun flag to world state, to solve problems with short missions. (#118, #236)
Fix: [BREAKING CHANGE] Rewards now sent as simple strings rather than XML, for speed - changes recorded rewards format. (#261)
New: ALEAgentHost.setSeed allows ALE experiments to be seeded. (#254)
Fix: No longer need a fresh MissionRecordSpec for each call to startMission. (#256)
New: [BREAKING CHANGE] MissionRecordSpec.getTemporaryDirectory() now moved to AgentHost.getRecordingTemporaryDirectory().
New: MALMO_TEMP_PATH environment variable now used to determine where temp recording files are created. (#21)
New: TimestampedFrame now includes xPos,yPos,zPos,yaw and pitch information. (#257, #250, #231)

Which version?

• Unless you want to use the Arcade Learning Environment, avoid the _withALE builds.
• If you are on Ubuntu 16.04 then you can use the 15.10 release.
• The Redhat build was made with Fedora23 but may work with other Redhat flavors.

Breaking changes:

• If you are parsing the rewards.txt in the saved .tgz mission record, then you will need to change the parser to use the new non-XML format of dimension:value pairs as comma-separated <int>:<float>.
• If you are using MissionRecordSpec.getTemporaryDirectory() then you will need to change it to be AgentHost.getTemporaryDirectory()

Some necessary changes to the ALE wrapping.

Changes:

New: DiscreteCommandHandler now supports AgentQuitFromTouchingBlockType and RewardForTouchingBlockType. (#241)
New: MissionSpec has get/setSummary, getListOfCommandHandlers, getAllowedCommands. (#217)
New: DiscreteCommandHandler now supports attack and use commands. (#219)
New: Bonus - added AllowedMobs to ServerInitialConditions to control which mobs are allowed to spawn (does not affect mob_spawners)
New: Bonus - can now draw mob_spawner blocks and specify the entity they spawn as a variant.
New: ObservationFromRay returns info about nearest block/item/entity in the agent's direct line of sight. (#184)
Fix: RewardForDiscardingItem/RewardForCollectingItem now work with variations, colours etc.
Fix: Inventory and Entity observations now return information in the same format we use to specify things (eg Type, Variation, Colour)
New: More types supported in BlockDrawing, Inventory initialisation etc - see FlowerTypes, EntityTypes, MonsterEggTypes
New: AgentQuitFromCollectingItem (#171)
New: ObservationsFromFullStats now additionally returns Air (#214)
New: Cross-platform Human Action Component

Which version?

• Unless you want to use the Arcade Learning Environment, avoid the _withALE builds.
• If you are on Ubuntu 16.04 then you can use the 15.10 release.
• The Redhat build was made with Fedora23 but may work with other Redhat flavors.

Let us know if you need more builds.

Breaking changes:

• MazeDecorator waypoint items have changed:

Previous syntax:

<Waypoints quantity="10">
  <WaypointItem>cookie</WaypointItem>
</Waypoints>

New syntax:

<Waypoints quantity="10">
  <WaypointItem type="cookie"/>
</Waypoints>

This allows us to add variants, colours etc - eg, to add spawn eggs of various types:

<Waypoints quantity="10">
  <WaypointItem type="spawn_egg" variant="Creeper Skeleton Spider Ozelot"/>
</Waypoints>

• ObservationFromHotbar, ObservationFromFullInventory, ObservationFromNearbyEntities etc now return data in the same format as Types.xsd

eg instead of returning a string like "item.dyePowder.red", they will now break the data down into the types / variants / colours found in Types.xsd, and return them separately.

This means that data is now returned in the same format that we send it (eg in DrawItem, InventoryObject, RewardForCollectingItem, etc.etc.)

Deprecations:

• InventoryItem and InventoryBlock have been deprecated in favour of a single new entity, InventoryObject. Existing inventory XML will continue to work, but InventoryObject is now the preferred way to go. It avoids having to make a distinction in the inventory code between blocks and items, and also allows item variations and colours to be specified.

Changes:

New: Added ObservationFromNearbyEntities (#89) and updated reward_for_items_test.py to demonstrate it.
New: Simplified inventory movement with swapInventoryItems command (#148).
New: Can combine inventory slots using new combineInventoryItems command (#189).
New: Free-floating blocks can now be placed using DrawItem, and used in RewardForDiscardingItem/RewardForCollectingItem.
New: Crafting now triggers RewardForDiscardingItem/RewardForCollectingItem.
New: Python bindings changed for videoframe pixels - now returns a native python array, for faster use in numpy/PIL etc. (#187)

Which version?

• Unless you want to use the Arcade Learning Environment, avoid the _withALE builds.
• If you are on Ubuntu 16.04 then you can use the 15.10 release.
• The Redhat build was made with Fedora23 but may work with other Redhat flavors.

Let us know if you need more builds.

Breaking changes:

• Inventory commands are now more sensible. Previously:

selectInventoryItem 3
dropInventoryItem 30

Now:

swapInventoryItems 3 30

• Python image retrieval is now simpler (and much faster):

for frame in world_state.video_frames:
    image = Image.frombytes('RGB', (frame.width, frame.height), str( frame.pixels ) )

New: Basic, first stage crafting support added (issue #11).
New: ObservationsFromMazeOptimalPath has been turned into general purpose ObservationFromSubgoalPositionList
New: Maze generator can now take care of quitting mission when agent reaches goal (issue #103)
New: AgentQuitFromReachingCommandQuota (issue #109).
Rewards are now only sent when triggered (issue #120).
Multi-dimensional rewards are now possible, using a 'dimension' parameter on each RewardProducer.
Pitch command had positive as up, which was inconsistent with Minecraft and Placement section.

Breaking changes:

• Rewards that were previously accessed like this:

for reward in world_state.rewards:
    print reward.value

are now accessed like this:

for reward in world_state.rewards:
    print reward.getValue()

• In the .tgz saved mission file, rewards are now saved as XML rather than JSON.

New: tp x y z command (issue #112)
Fix: Hotbar commands moved from ContinuousMovementCommands to InventoryCommands
Fix: Client quit producers are now queried between executing commands, in cases where commands get clustered.
New: Test for AgentQuitFromReachingPosition
Some MissionSpec calls now take float coordinates to match XML (issue #107). MissionSpec::endAt now takes tolerance parameter.
peekWorldState and getWorldState now return WorldState instances rather than smart pointers (issue #124).
New: API call MissionSpec::setViewpoint to change the camera viewpoint.
Fix: ALE_HAC.py was broken (issue #114).

New: Issue #55 (added forceReset flag to FlatWorldGenerator, DefaultWorldGenerator and FileWorldGenerator).
New: Issue #84 (added seed to FlatWorldGenerator and DefaultWorldGenerator).
Add: Issue #82 (new RewardForDiscardingItem mission handler).
Add: Issue #73 (added new DiscardCurrentItem command to InventoryCommands).
Add: Issue #4 (Overclocking) - can set tick length, and limit onscreen render. (Required XSD change, hence bumping to 0.12)
Fix: Issue #48 (Grid observer crippled by bad TCP string sending method).
Fix: Issue #52 (Malmo's internal Minecraft client/server messages limited to 64k data per message).
Fix: Issue #40 (added tostring() methods to Lua and Python classes).
Fix: Issue #7 (requesting depth map would break the recorded video).
Fix: Issue #81 (changing video size between missions could crash the platform).
Fix: Issue #2 (XSDs no longer need to be in the current folder).
Add: New API call: MissionSpec::requestVideoWithDepth.
Add: New API call: AgentHost::peekWorldState.

N.B. This release contains a bug in ALE_HAC.py. This is fixed in master and will be fixed in the next release.

Fix: Issue #22 (attack auto-repeat not working if Minecraft hasn't had focus).
Fix: Issue #33 (framerate slowdown after 500+ missions).
Fix: Issue #37 (occasional messages lost because two servers listening on same port).