With May coming to a close, I would like to start the first of a monthly feature - some explanations of what’s going on under the surface of OpenSim. But wait, you may ask; doesn’t Justin already provide excellent weekly reports on what’s been committed? Of course - where this differs is in the level of detail, particularly on infrastructure projects. Justin does a weekly feature which means he only has time and space to report on features in bullet form. Where I plan to differentiate with these posts is in the details - not only what has been committed, but also where it is going.

There has been a lot of progress this month - starting at revision 9358 there have been several major areas of focus.

Networking: LLUDP Improvements

We now support the Multiple Object Update packet format for Linden UDP clients (most notably Second Life & realXtend), this builds on work that Mikko of realXtend provided late in April that did the same for Avatar Updates. The work done by Melanie reduces the total number of packets that need to be tracked by packing multiple updates into each packet - resulting in a significantly lower workload for our packeting code. The Avatar packet aggregation with 100 Avatars in the region results in a massive 71% reduction in the number of packets transmitted, and a 29% drop in CPU requirements, according to tests done by Intel for ScienceSim. Object aggregation produces similar results (although hasn’t been measured precisely.)

Packet Pooling - the next major improvement is in ‘packet pooling’ - one of the drawbacks to managed languages is very little control over optimising garbage collection. OpenSim handles potentially thousands of packets each second, in our previous packet system for each of these packets, we created several classes representing the internal data structures. (eg, Byte[] Packet becomes MyPacketObject) - while the processing is fairly quick, creating these packets results in the Garbage Collector handling tens of thousands of tiny unused classes at each collection. On .NET this runs so-so, but on Mono it runs atrociously.

Packet Pooling solves this problem by recycling classes - rather than leaving a dead class to be garbage collected, we stuff new data into it and re-use the class instance. We previously did this work back in 2007 with the help of 3Di developers, however it was at the time proven unworkable due to problems with old data persisting into new data blocks. Melanie managed to solve this issue, and nearly two years later we have it re-enabled. Early pooling code did not show much promise past 40 avatars (although up to that point was fairly dramatically better) - however more profiling done by intel managed to locate the problem as being due to a finite sized pool - a patch fixed this problem and the overall improvement was a further 40% reduction in CPU requirements for 100 avatars (total reduction for this and aggregation is a whopping 57% reduction in CPU requirements for a busy sim).

Overall, the OpenSim networking code has been improved very significantly (especially with April’s texture resend fixes), and I would appreciate seeing more testing with large userbases - the Wright Plaza weekly meetings have been running better than they have for a long time, but only represent a peak of 35 concurrent users.

Some early bugs did present themselves with this networking code - most notably running into several MTU limits for international users (apparently most international trunk lines will simply drop rather than fragment big UDP packets), myself, Nebadon and Melanie managed to identify and fix this problem.

I introduced, then disabled support for NAT Loopback - this allows you to emulate a loop-back compatible router for those hosting from home and unable to get a public internet IP. This is currently commented out due to issues with compatibility in Grid Mode (however Standalone mode can be made to work effectively) - further improvements will be in the realm of adding UPnP support allowing you to host a region from home reliably. Finally I also introduced the capability to use IP and DNS based bans (including wildcards), Dr Scofield improved this with some stability checks.

Groups Support

mcortez has been working on adding a fully functional implementation of groups into OpenSim this month - previously located as part of the flotsam project, the groups implementation has hit core. Early revisions ran into a problem with a permissions exploit and group deeding (allowing anonymous users to group deed a parcel), however these have been fixed and it is ready for further testing. My initial feedback is that it looks good - things such as group chat are behaving well and responsively, although it is not yet fully functionally complete yet (particularly around the area of group permissions).

New Region Modules Framework

I’m not entirely sure if this was introduced this month - however a large amount of work was done porting from IRegionModule to the new ISharedRegionModule interface[s]. This new version is syntactically nearly identical, however has better support for removing regions at runtime, and also utilizes the Mono.Addins framework, which will be essential as we begin to add PECL/CPAN-style module repositories to OpenSim (although exactly when this happens is an unknown). The older region module format is going to remain working for quite some time yet - however if you are feeling adventurous you can investigate the new format and look at porting modules across.

New Services Framework

As part of a general improvement of OpenSim code quality, Diva and Melanie improved the services framework this month, deleting the old OGS1 ‘UGAIM’ in favor of a new ‘OpenSim.Services’ binary - this also means that the Grid mode and Standalone mode now share all the major infrastructure internally (Standalone runs the grid services in ‘private’ now) - this reduces code complexity and makes it easier for us to add and adjust the services framework.

This framework has also made it easier to edit and adjust module points for Grid Services, meaning it is now possible to write Grid Service plugins - such as asset caches, external login systems, etc. Diva has been utilizing these to improve the functionality and quality of the Hypergrid technology. A new cache was implemented using GlynnTucker cache - after it was discovered the previous asset cache did not infact, cache anything, early testing indicates this one works.

Finally, Arthur implemented some improvements to the User Service framework which prevents people from seeing the ‘You are already logged in’ message’ in grid mode.

Remote Control, Administration and Web Integration

BlueWall introduced and improved his JSON Stats plugin system for OpenSim which allows remote monitoring via AJAX applications of a region. Melanie introduced a ‘remote console’ system that utilizes REST, and I introduced some basic capacity for Grid Owners to be able to send grid-wide announcements for maintainence purposes. Dr Scofield improved the XMLRPC-based Remote Admin module to handle additional formats. All of these will no doubt go towards improving administrator remote tools and web interfaces.

Inventory Archives

Justin continued work on Inventory Archives - these allow you to save a copy of your entire OpenSim inventory to a file on disk, which you can then restore on any other compatible OpenSimulator instance. For those who work on multiple seperate OpenSim instances (such as a personal grid, public grid, etc) being able to synchronise your inventory between the two is a big help. IAR archives are still highly experimental and should not be relied on yet for any production purposes.

Justin also added the ability to store creator IDs as a URL rather than a UUID - this allows you to reference foreign grid avatars as a creator.

Prim Meshing

Dahlia continued work on the internal ‘Prim Mesher’ used for physics. Improving sculpty physics shells to be more accurate and support properties such as Mirror, improving the overall meshing performance and adding the capability to disk-cache converted meshes for faster startups.

XML Serialisation Formats and Region Archives

Sean and Justin made several improvements to our XML Serialisation support including adding unit tests to the serialisers. This will improve overall reliability and support for OAR files as we move forward (preventing OARs from becoming version-incompatible with each other.), OARs were also improved with support for a wider variety of .tar formats courtesy of Dr Scofield. Finally, OARs were renamed from .tar.gz to .oar as the default filename.

Scripting

I improved MRM scripting, both the Engine and the API. The engine has been improved to support real-time loading of external binaries, including debugger support. This when compiled with the MRM Loader project (located on the forge) allows you to write (and debug!) MRM scripts from within Visual Studio (very cool.), MRMs were also improved with additional security checks, fixing of certain properties (such as Avatar.Position), events (World.OnNewUser) and introduction of an Audio API (World.Audio and IObject.Sound).

LSL was improved with support for the LSL Webserver API going into production on the Second Life(tm) Beta Grid. Also implemented was llAttachToAvatar, llDetachFromAvatar and a host of other improvements.

IRC Clientstack

As mentioned earlier, OpenSim now has a ClientView which supports a primitive text-only IRC protocol support. This allows you to connect to a region for chat facilities using only an IRC client - this may be an advantage over the previous IRC Bridge module where IRC users were not represented inworld with an avatar.

Final Words

If I missed anything - let me know, Justin’s notes are likely to be slightly more comprehensive on smaller non-architectural changes.

OpenSim is now a semi-RFC1459 compliant IRCd. IRC clients are treated as full members of the virtual world, gaining a presence with a horrible default avatar, as the screenshots below picture. Each IRC client has an implementation of IClient* which means we could in theory insert text descriptions of the world and any events in. (Although in practice it’s unlikely to be useful for anything beyond basic text & IM messaging with an IRC client).

The advantage of this over the IRC Bridge module is that each presence is visible in both environments (users show up in the ‘who’ list on IRC, and have an avatar ‘in world’), and we will be able to handle a number of events more directly and appropriately - such as private messaging or group discussions. The IRCd framework I have written is extremely basic and probably not 100% to spec - if anyone with IRCd experience wants to get into the code and fix it up, please do.

In the mean time, enjoy. You can enable this by using the [IRCd] Enabled=true option in OpenSim.ini and connecting on port 6666.

Fig 1. XChat

Figure 2. Hippo Viewer

Every morning as part of my ‘get started’ routine, I check most of the OpenSim websites, plus technorati and google searches for OpenSim news. I like to keep informed as to what is being developed, and who is putting OpenSim into production. The problem is it takes a good 15-30 minutes to trawl everywhere.

RSS is the obvious solution - unfortunately every single RSS reader for Windows seems to be stuck with an identical UI - a UI based on Outlook’s terrible RSS reader, which I happen to think makes every attempt at making casual reading as disinteresting as possible. In short, they all suck.

I have a iMac on my desk for testing - and on there I have installed Times. Times is an attempt to make a virtual newspaper built from RSS content - for the most part it works pretty well, but has a few really big drawbacks - it requires OSX (which I rarely boot into for normal use), it’s also uses way too much eye candy - viewing a single post gets half of the text area obscured with a silly ‘bent page’ effect.

Microsoft apparently made a similar application called Codename: Max - unfortunately it was discontinued somewhere after WinXP SP2, and is no longer installable on any system (trust me, I tried). So for the last few weeks I’ve given up and just keep searching manually - until this morning when I decided I’d built my own.

Wordpress has some pretty nifty ‘magazine’ themes, which do automatic translation of blog content into a newspaper-style viewing. Combining this with WP-o-Matic which turns RSS feeds into posts automatically; I have a pretty cool setup going. It took about 3 hours to install and customise (I ended up adapting the ‘Magazine-Basic’ theme with the TTF Titles plugin and stripping out a lot of the navigation) - and then I inserted all the feeds from Planet OpenSim, Technorati, etc.

Overall it works pretty well - I have it configured to load all the OpenSim news, and it shows up in a nice chronological listing (screenshot above is scrolled back about 30 pages into late January). The layout is fairly consistent and easy to look at and being Wordpress & PHP, pretty easy to customise further.

This weekend, I have been experimenting with replacing OpenSimWI with open source social networking software - Elgg. For those who haven’t used or seen Elgg before, it is very similar to things like Facebook - user profiles, groups, etc.

The key with what we are aiming to do however, is eliminate the duplicated data. That is, if you have a group “inworld”, the exact same data should be used on the website. Join a group on the website - it should show up in world. The way we have done this has been to rewrite the Elgg database layer so that it reads and writes from the same databases the OpenSim Grid Servers do.

The result is you can participate in the grid, without needing to be logged in. Grab an RSS feed of your group notices or send someone an offline IM from the OSGrid website.

Custom Profile Pages (Editing)

A Sample Profile

Friends List (Detail)

Group Listings (Popularity Tab)

Group View

When?

There is no deadline for when this will be launched, but we’re hoping to do so within the month.

If you have taken a look at the OpenSimulator mailing lists or IRC channels in the last 24 hours, you will have found a link towards a new ‘feature tracker‘. It works kind of like Mantis for bug tracking, but it is optimized for quick and simple feature request posting. I should however make some notes on how this works.

First, this is not a roadmap. Putting a feature up and getting a high number of votes won’t get it implemented - however a developer looking at the idea and saying ‘Hrrm, I could do this’ will. High votes and high views will make it more likely that a developer looks at your idea (and may influence priorities), but it is not binding.

Second, this replaces mantis for feature requests - our mantis install has traditionally been the place feature requests go, but a lot of developers havent wanted them there since it is officially a bug tracker only. This tracker seperates the two out and recognizes features needing different tools.

It’s a work in progress and fairly experimental (I wrote the whole thing from scratch in an afternoon), but go take a look and post your OpenSim suggestions.

It’s time for some more pretty pictures to help answer a debate between myself and fellow OSGrid admin Nebadon. The core question is essentially; “Have the users who joined us since the LL OpenSpace price change, stuck around?”. The methodology we are using is based on last login times - it’s the only bit of data we store relating to how active users are, that we can query in a reasonable period of time.

The three questions we will ask of the database are; “How many unique users logged in in the last day? week? month?” - of the 10,958 who have at least one recorded login in the last 6 months since we added the field, 468 logged in today, 1,142 this week and 2,822 this month. The next question to ask is - how does this break down across by signup date? The following highly compressed chart shows the information for the number of logins in the last 24 hrs, 7 days and 30 days from each signup group. (A signup group is all the users who signed up in a certain calendar week).

Fig 1. Logins versus Signups

You can still pretty distinctly see the point at which Linden announced OpenSpace price changes (Y2008W41), but it doesn’t quite explain if these users are sticking around more, less or the same as users who signed up beforehand. We’re expecting some kind of drop-off since a lot of users were just experimenting with alternatives - and we’re not really focusing on being a SL clone (unlike some of the other grids). So, the next chart shows the above data - but divided by the total number of signups in that week.

Fig 2. Signups versus Logins Proportionally

I clipped off some of the earlier data since it was giving erroneous values, but what is interesting here is that older users have a fairly incredible retention - >30% of users who signed up in 2007 / early 2008 are still around and logging in. Newer users have dropped off - but there’s still a pretty high proportion who stayed - both myself and nebadon believe these might be region operators - early users were all region operators, and region operators probably have a pretty good retention.

So let’s take a look at some more data - the following is logins from the last 30 days - how many are region owners (cross referenced against signup date)

Fig 3. Region Owner versus Logins

Now this is a little incomplete, since it wont measure things like estate managers - it only shows people who are the registered region owner by the Master Avatar parameter. There is a trend for people with older accounts who still login regularly to be more likely to own a region - but it’s not absolute.  Looking at this data you might be mistaken for thinking that newer users are less likely to own a region - but charting the data % of region owners by signup date you get a different picture.

Fig 4. Proportion of Region Owners

In conclusion - the data seems to indicate mixed results to the original question; users who signed up looking for an SL alternative are more likely to have left than users who joined beforehand - but trends seem to be indicating the core underlying growth is still pretty strong; and a healthy percentage of the people who did sign up in that period have stuck around - overall growth of active users increased by between 200 and 600 percent after that jump.

There is also a trend that the more active users are region owners - of the people who logged in in the last 24 hours (versus 30 days for Fig 3), closer to 50% were region operators. This doesnt strike me as too suprising given that OSGrid’s real strengths are in the ability to connect in regions free of charge - more active users are more likely to b involved in that aspect of the grid.

For other grid operators who are interested in calculating the same numbers I have above for comparison, on the next page are the horribly complex SQL queries I have been utilizing to generate this data. They should work with any OpenSim-standard MySQL installation. Click here to see them.

This is modelled somewhat after the same guide for wordpress. Running an OpenSimulator instance is a challenging prospect for many - there are a number of important criteria you should be looking at before purchasing, but the single most important question is “Will I be managing this myself?”.

“Do it yourself”

DIY Hosting OpenSim while relatively easy by comparison to most hosting software, is still not something that should be done by the technically uninitiated. OpenSim requires a healthy amount of maintainence in the form of updates, backups and patching.

Virtual or Dedicated?

For DIY hosting you have two choices in terms of hosting - Virtual Private Servers (VPS) or Dedicated Servers. Virtual Servers are a percentage of a physical server provided inside of a virtual machine. Virtual servers are often significantly cheaper than a dedicated server, however come at a heavy cost in performance.

Virtual server pricing starts at approximately US$15.00 per month. Dedicated servers will start at approximately US$50.00 per month. Virtual servers are appropriate for low performance private sandboxes, Dedicated servers are a must for public or regular usage.

A final note on dedicated servers - the most inexpensive dedicated server will often outperform the most expensive VPS. If you are on the price border between the two, I recommend switching to the lower performance, but dedicated server - than the higher performance but shared VPS.

Linux or Windows?

Unless you have significant investments in using Linux for your hosting, I recommend using Windows 2003 or 2008, especially if you are not familiar with running hosting environments. The reasons for this are fairly simple - Mono has memory management problems, memory requirements under Mono are often doubled or quadrupled.

Unfortunately Windows server is not always availible - especially at the lower price ranges. Microsoft charges hosting companies a $10/month per cpu fee for Windows Server, so this will always be passed on in the form of higher prices. In the highly competitive VPS industry this price increase makes good Windows VPSs a rare niche.

What processor?

OpenSim will very rarely peg your processor - I would not bother investing too much into the processing capacity of your instance or server. (Whether a VPS or Dedicated machine).

My own experience says that “off the shelf” desktop processors are more than adequate. Here at DeepThink we have standardised on Xeon 3220 based machines (Quad Core, 2.4Ghz) and we find these are more than suitable for our higher-end clients. For an average user, a off-the-shelf Core2Duo is more than sufficient.

How much memory?

This is a much more relevant question for OpenSim. If you are running on Mono, I recommend no less than 1GB of memory per region (512MB if it is mostly empty). If you are running Windows/.NET you can use a rule of thumb at approximately 384 to 512mb per region.

A more precise metric is to total up the following: A base cost of 200mb, total size of all textures in the region + 30% in megabytes (average region is around 50mb), 30mb per 1,000 primitives, 50mb per concurrent user and double the sum if you are running under Mono.

So a region with 5,000 primitives, an average number of textures (50mb) and ten concurrent users you should allocate approximately 900MB of memory for .NET and 1.8GB for Mono. Leave at least 100mb on top of this sum for the operating system itself; if the machine runs out of memory page file access will cripple the performance of your region.

Memory should be as fast as possible - the faster you can get the better.

How much hard disk?

None. OpenSim rarely uses more than a few gigabytes of space unless you are operating a large grid (OSGrid.org is only about 120GB of assets currently). Anything over 10GB is sufficient.

How much bandwidth?

This is a slightly tougher question to answer - it depends a lot on your sim, the complexity and texture detail, the number of visitors. We at DeepThink sponsor the servers running the OSGrid.org Plaza’s, which I believe are the six most visited OpenSim instances running. The Plazas are broken up over two physical machines - each uses about 240GB of bandwidth each month - or approximately 80GB per region per month.

In pratice your region should come in significantly under this - but I recommend making sure you have at least 100GB of traffic per region in the event that you create a popular space. Most hosts provide at least 1TB of transfer on dedicated servers, so this should in most cases be a non-concern.

The other side of bandwidth is connectivity - how fast your server can speak with the outside world. I recommend having at least 200kbit per concurrent user. So for a standard ‘10 concurrent’ situation, you should have at least 2mbit of connectivity.

Backup Space

Make sure to have some kind of external backup system - most hosting providers will sell you SFTP access to a backup server, however make sure that they in turn provide good backup policies on your backups (offsite backup, etc.). OpenSim can be very fickle software, so make sure you have a good backup routine setup.

Latency

If you are a non-american user or cater to a foreign crowd, Latency is a very important issue. Latency is the “lag” between when you do an action and when it occurs on the simulation server - unfortunately latency is often best fixed by changing the geographical location of your servers.

If you cater to an audience in a specific country - I would recommend searching for servers in countries nearby. Often localised hosting is several times more expensive than in the US due to higher prices for local bandwidth, however most of the time there is a cheaper alternative nearby. For example, Japan has very high local server costs - but Korean or Singaporean servers are cheaper and have similar latency characteristics.

Who?

Dedicated Servers

CariNet (San Diego, CA, USA)

CariNet have been a popular choice for people running regions on OSGrid thanks to great pricing and astonishgly quick setup times. Their live support is pretty good and tends to fix most issues within 30 minutes of reporting. Cari have a highly flexible pricing system - so make sure to remember to include additional RAM and the Windows operating system with your purchases.

CariNet are also an official sponsor of OSGrid.org, providing regular free upgrades to the official servers that are hosted there (Currently the two plaza machines and the one welcome machine). They have also provided some special pricing for OpenSim users - see the italics below.

Pricing: Minimum $50.00 per month (special offer at time of writing).

I spoke with CariNet’s sales department, and we have worked out a few “OpenSim Optimized” configurations -

• High Performance - Windows, 4 Core Xeon, 4GB configuration is US$200.00/mo with no setup fee.
  This configuration is ideal for a “SL-replacement” style region host, you could probably get a couple of regions off this hardware, but the exact number will vary as above. I would estimate approximately 2 “Full Regions” could be run on this hardware configuration.
  If you want this server for only $160/month (but with a $120 setup), contact Cari.net sales via their Live Chat with the discount code “OSGRID”. (Speak with Mike or Shawn)
• Mid-Range - Windows, Core2Duo, 2GB configuration is US$140/mo with no setup fee.
  Expect to get one heavy region out of this configuration (or quite a few “homestead-esque” regions) - as always, this varies based on consumption.
  As above, if you want this server for only $120/month (but with a $60 setup), contact Cari.net sales via their Live Chat with the discount code “OSGRID”. (Speak with Mike or Shawn)
• Low-End - Windows, Celeron 2 Core, 2GB configuration is US$125/mo with no setup fee
  Consider this a homestead replacement, you should be able to get a small number of “homestead-equivilents” running off of this - obviously you dont have primitive limits or things like that, but you will notice degrading performance if you rely on a lot of scripted functionality.
  As above, if you want this server for only $105/month (but with a $60 setup), contact Cari.net sales via their Live Chat with the discount code “OSGRID”. (Speak with Mike or Shawn)

Annual and quarterly discounts are availible for paying more than one month at a time.

Website: http://www.cari.net/

OVH (Paris, France, EU)

OVH are the largest dedicated server host in Europe and provide unbeatable pricing on European servers. English pricing is expensive, however the same servers ordered from their French or German language sites are 10-20% cheaper.

OVH provide large amounts of memory and disk space by default, so very little customisation is required on their default servers. Windows 2003/2008 is an additional EUR 15.00 per month. Their RPS low-end machines may be a suitable mid-way point between a VPS and a full machine, but I haven’t tested one myself.

Pricing: Minimum EUR 49.95 per month. An OpenSim optimized configuration will start at EUR 114.00 per month (fully equipped).

Website: http://www.ovh.co.uk

Virtual Servers

Tektonic (Dallas, TX, USA)

Highly recommended by one of our employees, Tektonic provide good quality Linux VPS solutions - while they only offer Linux based VPS instances, their pricing is affordable and do not appear to oversell their servers in any noticable capacity.

Pricing: Minimum $15.00 per month. A “workshop” quality VPS is US$28.00 per month.

Website: http://www.tektonic.net

Slicehost (St Louis, MO, USA)

Slicehost guaruntee a minimum level of performance on their virtual servers which makes them more expensive than other hosts, however have a solid customer service reputation and a high quality control panel.

Pricing: Minimum $20.00 per month. A “workshop” quality VPS is US$38.00 per month.

Website: http://www.slicehost.com

FsckVPS (Atlanta, LA or Texas)

FsckVPS is often recommend on the OSGrid forums as a potential host - their slices are of a reasonable configuration and have plenty of memory availible. Processing time is more limited and some users have complained about running into processor limits, so their higher end packages are recommended.

Pricing: Minimum $9.95 per month. A “workshop” quality VPS is $19.90 per month, however I recommend the $34.90 plan if you plan on concurrent users.

Website: http://www.fsckvps.com

“Someone host it for me”

As an alternative to setting up and managing your own OpenSim instance, you can pay for someone else to manage it for you. Often this is simply bundled with a grid - however a number of companies (including DeepThink) offer managed hosting services for OpenSim.

On average, an OpenSim instance will take around four hours of maintanence each month - at an average industry salary this is around $80.00 per month to the hosting company in engineering costs. You should expect the margin per-server to be about this size.

OpenSim hosting companies will charge based on one of two methods - the first method is Second Life(r) style limits - Primitive, Avatar and Script limitations. The companies that charge this way are generally those providing regions within a grid.

The second method is charging for the underlying service used - Processor time, RAM usage and Bandwidth. The companies offering this tend to be fairly agnostic as to where your regions will connect.

An important distinction between the two methods is that you will often get more ‘bang for buck’ on the underlying capacity. For instance if you need multiple low performance regions - you will come out significantly ahead by purchasing server time rather than regions. Likewise if you need a single high performance region you can tailor your requirements more specifically.

A final note - many simulators are sold as ’supporting 45,000 prims’. This is not reflective of actual usage - the 45,000 number is the largest number the Second Life(R) client will display, however on the backend there is often no limit. That however does not mean the region is capable of supporting 45,000 prims.

We often find that simulator performance degrades rapidly past the 10,000 primitive limit per region and it requires a significant amount of work to get more than 15,000 prims into a region while the region itself is stable, especially when scripts and multiple users are factored into the equasion.

As an addendum to the above - the per region count is not reflective of the per-simulator count. OpenSim can host multiple regions per simulator instance and we have found that the simulator instance itself can host in excess of 100,000 primitives as long as they are distributed between multiple regions.

A note on SLAs and Uptime

OpenSim is still alpha software. If you expect 99%+ uptime, you will also expect to pay for it. Keeping a region running with guarunteed uptime requires active maintainence and attention by an engineer. This means it is impactical to expect 99%+ uptime unless you can afford a technician 24/7.

Our experience with client projects is between 97.6% and 99.2% uptime over the course of a month. (including scheduled and unscheduled maintainence) - this will vary slightly depending on the version of OpenSim used, number of users and other factors.

If someone claims to offer bug-free and stable deployments of OpenSim, they are probably lying or are very naieve. OpenSim will occasionally run into bugs. If you have a specific project in mind that requires a production deployment it is strongly recommended to hire a consultant who is familiar with the OpenSim project (eg DeepThink, IBM, 3Di, Tribal Media, etc.).

List of companies providing OpenSim hosting (note this is incomplete - there are likely to be others)

Customized OpenSim Hosting Providers

The following companies provide customisation services and will tailor your OpenSim install to your specific needs. These companies all provide pricing on a per-server basis rather than per-region.

DeepThink (Managed OpenSim hosting and Customisation services): http://www.deepthink.com.au

ReactionGrid (Managed OpenSim hosting): http://www.reactiongrid.com

3Di (Customisation services, hosting. Tokyo based.): http://www.3di-opensim.com/en/

Not listed here, but likely to offer these services are IBM, Tribal Media and Genkii.

Grid Hosting Providers

These companies provide hosting within the context of a grid. Most of these companies use very similarly specced hardware, however many of them share multiple clients per server - to use Second Life as an example, Linden Lab charges $295 per “island” which is stacked at 4 per server (or $1,180 per server per month).

While I did attempt to compile an exhaustive list of companies here - finding pricing and stacking ratios for many of them proved a difficult and time consuming process. You can find a more comprehensive list of grids (most of which offer region hosting), here:

http://opensimulator.org/wiki/Grid_List

Stability Remarks

As noted all over the OpenSimulator website - OpenSim is alpha quality software. To quote our downloads page

Please note: As OpenSim is still at an alpha code maturity stage, there is absolutely no guarantee that functionality works or is stable, even in the numbered releases. Certain features may not work either because the code is in rapid evolution, or because functionality expected by the Linden Labs Second Life viewer has simply not been implemented yet. However, constructive feedback is still welcomed.

If you are putting OpenSim into a production environment, make sure to speak with your hosting provider about what you plan to do - many of them may be able to make recommendations or tweaks to better suit your demands. If you have clients involved - it is often better to stick to an older more stable release as all releases are not nessecarily equal (eg 0.5.8 was somewhat stable, whereas 0.6.3 had appearance bugs).

If you are planning on using the ‘bleeding edge’ version of OpenSim in production, expect to get cut, or again quoting the downloads page:

There Be Dragons Beyond This Point

If you are truly feeling dangerous, adventurous, or want to help us test the next version of OpenSim you are welcome to grab the latest unstable code out of our subversion trunk. Any warnings previous expressed about the alpha nature of the code go double or triple if you are running directly off of trunk. Never, ever, ever, never run this in production environments, it is not suitable for that unless you are very familiar with the source code, and can hot fix any piece of it (that probably means you are an OpenSim core member). Feedback and testing on the unstable tree is appreciated, as that helps us make the next release better. If this scares you from using trunk, that was intended.

If it breaks, you get to keep both pieces.

Final Remarks

Running an Opensimulator instance can be a perfect fit for your organisation or person as long as you respect the limits. OpenSim can provide fantastic discounts to other commercial virtual worlds software, however it is worth remembering that it is still new and experimental - often this can be an advantage in customisability and features, but at the same time you need to respect the limitations of the software.

If you are unsure about whether to put OpenSim into production or how to do it in a way that fits what you need to do - like every Open Source piece of software, there are a lot of developers and organisations who can provide consulting services. Check the Core Developers List on the OpenSimulator wiki for a full list.

It’s currently 12:18 AM PST, and we’re sitting at 42 minutes away from shutting down the grid for the big asset conversion. I will be posting regular updates on the OSGrid twitter feed. Today we’ll be doing two important upgrades to OSGrid infrastructure - the first will be moving to new hardware for the asset server, the second will be converting to the new FragstoreFS backend.

The first step will be shutting down the grid so we can export the database onto the new server - based on previous exports, this is estimated to take approximately 2 hours. The next step will be importing the database and adjusting the schema for the conversion - this involves changing the indexes to add a numeric incremental index which allows us to resume conversion from specified points. The import will take around one to two hours, and the index conversion will take a similar period of time.

The next step will be firing up the fragstore converter - this is a specialised piece of software we wrote for this, which will reformat the data into the new CAS-aware schema, and move the BLOB records into the filesystem. Conversion speed varies greatly based on the size of the individual assets being processed, but we’re hoping this will take between 9 and 15 hours to complete - we preconverted a large portion of the database already; but the exact speedup we will get from the preconversion is unknown.

Total estimated downtime is 21 hours - but this will hopefully be the last time we need to shut down the grid for such an extended period of time, as future asset moves will be able to be done transparently to grid users (Murphy willing.).