|
|
Kindly hosted by
Distributed Loading of Data
Welcome to the tutorial about continous loading of data. I've created this tutorial so that some of you
can understand the principles behind uninterrupted rendering process, continuous loading of data
and, as an extra, multi-file-format saving/loading operations. So, let's start.
Table of Contents
Introduction
You probably heard about Dungeon Siege' seamless process of loading data for the upcoming territories
without ugly loading screens. Well, I'm presenting you with the base of such system. I'm saying 'base', because it
still requires more coding/improving/updating, so that you can seamlessly integrate it into the engine/game.
For example, you'd need a timed distribution (e.g a kernel) so that you can call these classes to do loading
in free time that's left after rendering current frame, so that the game still has a fixed frame rate.
Base File and Binary File classes
Firstly, we'd need a base class, the prototype for other derived classes such as Binary and Text files:
class CBaseFile
{
public:
CBaseFile();
virtual ~CBaseFile();
enum
{
File_BinaryFormat,
File_TextFormat,
File_XMLFormat
};
enum
{
File_ReadOnly,
File_ReadAndWrite,
File_WriteOnly,
File_Append
};
enum
{
File_Operation_Incomplete,
File_Operation_Error,
File_Operation_Complete
};
enum
{
File_Offset_Start,
File_Offset_Current,
File_Offset_End
};
virtual int Write(void *pvData, UINT uiDataSize, UINT uiBytesToStream,
long lOffset = -1, UINT uiRefLoc = File_Offset_Current) = 0;
virtual int Read(void *pvData, UINT uiDataSize, UINT uiBytesToStream,
long lOffset = -1, UINT uiRefLoc = File_Offset_Current) = 0;
virtual int Close();
virtual int Open(const char *pcFilename, unsigned int uiRWMode = 0) = 0;
virtual int Flush() = 0;
virtual bool IsOpen();
protected:
int m_iFileMode;
void *m_pvFileHandle;
char *m_pcFilename;
};
As you see, this class exposes the major functions that are going to be overriden by Binary and Text File classes.
The member variables, m_pvFileHandle is the generic abstraction for file intput/output (e.g stream) and the
m_pcFilename is self-explaining. The m_iFileMode stores the type of the file input/output - read only, write only etc.
And the various enum's will help us more clearly use the base class. Now, let's take a look at the Binary class:
class CBinaryFile : public CBaseFile
{
public:
CBinaryFile();
virtual ~CBinaryFile();
virtual int Write(void *pvData, UINT uiDataSize, UINT uiBytesToStream,
long lOffset = -1, UINT uiRefLoc = File_Offset_Start);
virtual int Read(void *pvData, UINT uiDataSize, UINT uiBytesToStream,
long lOffset = -1, UINT uiRefLoc = File_Offset_Start);
virtual int Close();
virtual int Open(const char *pcFilename, unsigned int uiRWMode = 0);
virtual int Flush();
unsigned int m_uiBytesStreamed;
bool m_bOperationComplete;
UINT GetCurrentOffset();
};
Obviously, this class overloads the Read/Write pure functions of the CBaseFile and implements them in the
following way:
int CBinaryFile::Write(void *pvData, UINT uiDataSize, UINT uiBytesToStream, long lOffset, UINT uiRefLoc)
{
if(pvData == NULL || m_pvFileHandle == NULL)
return -1;
if(uiDataSize >= m_uiBytesStreamed)
{
if(uiBytesToStream == 0)
return File_Operation_Incomplete;
if(lOffset == -1)
fseek((FILE *)m_pvFileHandle, GetCurrentOffset(), uiRefLoc);
else
fseek((FILE *)m_pvFileHandle, lOffset, uiRefLoc);
void *pvOffset = (char*&)pvData + m_uiBytesStreamed;
if(uiDataSize > m_uiBytesStreamed && uiDataSize < m_uiBytesStreamed + uiBytesToStream)
{
if(fwrite((char *&)pvOffset, uiDataSize - m_uiBytesStreamed, 1,
(FILE *)m_pvFileHandle) == 1)
m_uiBytesStreamed = uiDataSize;
}
else
{
if(fwrite((char *&)pvOffset, uiDataSize - m_uiBytesStreamed, 1,
(FILE *)m_pvFileHandle) == 1)
m_uiBytesStreamed+= uiBytesToStream;
}
}
if(m_uiBytesStreamed == uiDataSize)
return File_Operation_Complete;
return File_Operation_Incomplete;
}
int CBinaryFile::Read(void *pvData, UINT uiDataSize, UINT uiBytesToStream, long lOffset, UINT uiRefLoc)
{
if(pvData == NULL || m_pvFileHandle == NULL)
return -1;
if(uiDataSize >= m_uiBytesStreamed)
{
if(uiBytesToStream == 0)
return File_Operation_Incomplete;
if(lOffset == -1)
fseek((FILE *)m_pvFileHandle, GetCurrentOffset(), uiRefLoc);
else
fseek((FILE *)m_pvFileHandle, lOffset, uiRefLoc);
void *pvOffset = (char*&)pvData + m_uiBytesStreamed;
if(uiDataSize > m_uiBytesStreamed && uiDataSize < m_uiBytesStreamed + uiBytesToStream)
{
if(fwrite((char *&)pvOffset, uiDataSize - m_uiBytesStreamed, 1,
(FILE *)m_pvFileHandle) == 1)
m_uiBytesStreamed = uiDataSize;
}
else
{
if(fread((char *&)pvOffset, uiDataSize - m_uiBytesStreamed, 1,
(FILE *)m_pvFileHandle) == 1)
m_uiBytesStreamed+= uiBytesToStream;
}
}
if(m_uiBytesStreamed == uiDataSize)
return File_Operation_Complete;
return File_Operation_Incomplete;
}
As you may notice, both functions are almost identical, the only difference is the fwrite/fread words :)
We could probably unite them under one function and use bool bReadWrite flag - true for read and false fo write,
but hey, I prefer that functionality is separated into logical blocks.
OK, so we've got so far. In those two functions, the logic is: write the data with the specified size using given block
size, but if the next writing amount exceeds given data block size, do not go over it. If the total amount of written
data is equal to the data block size, return File_Operation_Complete. That's it!
Text and XML File classes
Well, the only difference here is that the Text file is going to convert read data into Text data. Nothing more,
nothing less. About XML.. I'm not sure how to go here myself, so wait until I think something out
about streamed loading of hierarchical data (XML?).
Continous Reading/Writing
To implement continuous reading/writing, as seen in project source, we have to create a task system, something
close to a scheduler or an engine kernel and assign him the tasks of loading the data in the intervals where
there's free processor cycles are left. For example, if we want to render scene at 60 frames per second, but we
got an ability to render it at 80, then why not lock the frame rate and use free CPU cycles to load the data we need?
So, I see the system working in a way like:
See if there's any CPU cycles left after rendering the scene If there are, then
Do we need to load any data? Yes we do. Calculate the amount of data we can load using free CPU cycles and load it immediately. If we need to pause loading process, do not appoint Read/Write task for current frame run.
If there's no spare cycles, wait for next frame Start from (1)
Yep, that's it. You can think of rest by yourself, add/use this code with your task scheduler and do whatever you
want to do with it. I, for example, will be using to stream in textures/terrain/sounds and just general resources
on the fly using my kernel/task class from Enguinity series on the GameDev.net website.
Extras
Oh yeah, I forgot about extras. Well, the extras are... Let me think... Extras... Ummmm... Oh, I remember now!
Don't forget to send me your thoughts/comments on this tutorial, which is, by my standards, is as easy as a pie.
You can easily extend this code for type-independent saving/loading of files, which I have already written. It can be
either binary, text or XML (or whatever other format you may think of).
|
