A group project for the Virtual Environments assignment during my MSc.

Team members:
Eleni Maria Stea, Domna Banakou, Hayden Lee, Eleni Kokkinara

Screnshots: here.

We used the 3D Studio Max for the modeling part and the XVR scripting language to create a project appropriate for the CAVE environment. Students, who are the main end users of the application, are given the opportunity to attend a virtual class during which they are able to put together or apart the basic parts of a human skeleton, using a tracker device. They are able to move around the virtual room and observe the skeleton from different view points, offering the sense of looking at a 3D x-ray. The entire background of the class provides them with useful information on how to complete their task. Since the application combines the teaching efficiency of a traditional class with the joy of a game, it could be used as a teaching supplement, promoting an intuitive understanding of the human anatomy.

Assignment aims and approach:

The primary objective of the application was to make the user interface as friendly and intuitive as possible. For that reason, the whole virtual environment was designed to satisfy certain requirements:
  1. The virtual environment should reminded the students that their task was not only an entertainment game but an alternative way of learning.
  2. The use of the system shouldn't have been complex or confusing, otherwise it wouldn't have been attractive to the end users.
  3. The users should receive feedback on when they were completing their task successfully or not. That way they would actively participate to the virtual course and been able to get a final result by themselves.
  4. The users should been able to interrupt and restart the application at any time.
Our approach to satisfy the previous aims was the following:
  1. The virtual environment was designed to look like a traditional classroom in order to satisfy the first aim.
  2. The headtracker was used for navigating around the classroom and only 1 button of the wand device is needed for selecting and moving the skeleton parts. A second button is used for the scattering of bones. Thus, the second aim is satisfied.
  3. We used two different colors (red and green) to indicate the correct and the wrong placement of a bone in order to satisfy the third aim.
  4. The user can press a button to scatter the bones everywhere at anytime so that the fourth aim is satisfied.
Implementation:

3D Studio Max Modelling

The scene and a number of objects of the virtual environment were created or modified in 3D Studio Max. First we constructed the virtual classroom where the model of the skeleton was placed. Then, we added some helpful and decorative tables to make the classroom more realistic.

For the skeleton model a lot of preprocessing was needed. Since it consisted of many parts, we first had to group them to reduce their number. We also had to minimize the number of polygons to optimise the performance of the application. Then, we exported each grouped skeleton part to separate .aam files, after writing down their positions in the 3DS model (see XVR coding).

The scene, the blackboards and the skeleton picture were modeled in 3D Studio and different textures were applied afterwards; we used only a few polygons in order to achieve a faster execution.

XVR Programming

In the begining, we had to load each object in the correct size and position relatively to the CAVE and set the camera position which was different for the CAVE and the non-CAVE interface.

In order to place the bones in the correct position to look like a full skeleton, we had to use the positions extracted from 3D Studio Max af- ter inversing the z coordinate with the -y. The size of each object was calculated comparing it to other CAVE examples; since we wanted to be able to change it during testing, we set the size of each object relatively to the scene size and we added two buttons that could increase or decrease a scaling factor, apply the scaling and print the scaling factor value to the XVR console as seen in Figure 8.

In order to scatter and place the bones in random positions we used a random value generator that calculates random distances from the initial position for each part when a button is pressed. This distance is always smaller than the room size so that the bones are always inside the virtual classroom as seen in Figure 1. The only bone that never been moved from its initial position is the scull.

For the interaction between the user and the environment, a simple vir- tual cursor was used. The users are able to move the 3d cursor in order to grab the bones and position them correctly. They are allowed to pick up a bone when its bounding box (see Figure 6) and that of the cursor intersect. When this occurs, the bone is highlighted to give feedback to the users regarding which bone they are about to grab as seen in Figure 4.

The correct placement of a bone is determined by its distance from the ideal position, which is the initial one when the skeleton is complete as seen in Figure 5. If the bone is placed near enough to the ideal location, the program snaps it into that automatically. A threshold is set in order to specify the maximum accepted distance from the initial position. This threshold can be changed the same way as the scaling factor. By modify- ing the threshold we can adjust the difficulty level of the task since a very low threshold value requires greater accuracy as seen in Figure 7.

When a bone is dropped somewhere in the scene, it is highlighted. Two different colors are used: red to indicate a wrong placement and green to indicate a correct one as seen in Figures 2, 3 and 4.

Screenshots:

Figure 1: The skeleton bones are scattered.





Figure 2: The user placed the torso in the correct position.





Figure 3: The user left the bone in a wrong position (red highlight).





Figure 4: The user left the bone close to the correct position.





Figure 5: The task is completed.





Figure 6: Skeleton bounding boxes.





Figure 7: Bone position accepted as correct although not for
very large threshold values.





Figure 8: The skeleton size is changed when interactively
changing the scaling factor.




Figure 9: The skeleton inside the CAVE.





Back to top



Code:

#include <Script3d.h>

//Button mask:
//  32 16 8 4 2 1
//   |  | | | | \-Red
//   |  | | | \-Yellow
//   |  | | \- Green
//   |  | \- Blue
//   |  \- Joystick button
//   \- Trigger

//GetTrackerButtons(X)

/* Set global scene parameters */
SET SCENE_FOV  = 60;
SET SCENE_NEAR = 0.5;
SET SCENE_FAR  = 1000;

var in_cave = true;

var lt0, lt1;

var room_obj;
var room_mesh;

var bboard_obj;
var bboard_mesh;

var bboardskel_obj;
var bboardskel_mesh;

var bboardex_obj;
var bboardex_mesh;

var ptr_obj;
var ptr_mesh;

var ptr_pos = [0, 0, 0];

var show_bbox = false;

var skel_scale = 0.005;
var bboard_scale = 0.006;
var bboardskel_scale = 0.006;
var bboardex_scale = 0.004;
var room_scale = 1;
var snap_thres = 0.05;

var picked_obj = -1;
var hl_obj = -1;

var skel_offs = [0, 0.8, -1.0];

var skel_part_pos = {
    [-0.012, 281.258, -7.303],
    [-0.012, 210.655, -6.371],
    [43.862, 134.749, -4.115],
    [-44.692, 134.749, -4.115],
    [15.634, 47.491, -12.756],
    [-15.658, 47.491, -12.756],
    [17.09, 121.581, -7.288],
    [-17.115, 121.581, -7.288],
    [43.653, 173.607, -13.665],
    [-44.488, 173.607, -13.665],
    [36.548, 218.049, -13],
    [-37.378, 218.049, -13],
    [18.845, 7.34, -3.191],
    [-18.87, 7.34, -3.191]
};

var skel_part_names = {
    "head.aam",
    "body.aam",
    "lhand.aam",
    "rhand.aam",
    "llleg.aam",
    "rlleg.aam",
    "luleg.aam",
    "ruleg.aam",
    "llarm.aam",
    "rlarm.aam",
    "luarm.aam",
    "ruarm.aam",
    "lfoot.aam",
    "rfoot.aam"
};
var skel_parts = Array(0);

function find_collision();
function target_pos(idx);
function correct(idx);
function update_skeleton();
function scatter();
function CameraMoveMouse();
function MoveControl();
function DrawGrid(col, size);

function OnDownload()
{
}

function OnInit(params)
{
    setLocalDir();

    SceneSetParam(VR_HEADTRACKER,1);   //need this line to run in the cave

    /* initialize light */
    lt0 = CVmLight();
    lt0.SetPosition([0.0, 3.0, 0.0]);
    lt0.SetDiffuse(0.8, 0.8, 0.8);
    lt0.SetSpecular(0.8, 0.8, 0.8);
    lt0.Enable();

    lt1 = CVmLight();
    lt1.SetPosition([0.0,3.0,0.4]);
    lt1.SetDiffuse(0.5, 0.5, 0.5);
    lt1.SetSpecular(0.5, 0.5, 0.5);
    lt1.Enable();

    var PosC=[0.0,0.0,0.0];
    if(!in_cave) {
        PosC= [0.0,0.6,3.0];
        CameraSetPosition(PosC);

        var InititialCameraDirection = [0.0, -0.10, -1.0];
        CameraSetDirection(InititialCameraDirection);
    }

    ptr_mesh = CVmNewMesh("data/ptr.aam");
    ptr_obj=CVmObj();
    ptr_obj.setScale(0.05);
    ptr_obj.LinkToMesh(ptr_mesh);

    /*bboard*/
    bboard_mesh = CVmNewMesh("data/bboard.aam");
    bboard_obj = CVmObj();
    bboard_obj.setScale(room_scale * bboard_scale);
    bboard_obj.LinkToMesh(bboard_mesh);
    bboard_obj.setPosition(0, 0.4, 0);
    bboard_obj.setRotation(90, [1, 0, 0]);

    /*bboardskel*/
    bboardskel_mesh = CVmNewMesh("data/bboardskel.aam");
    bboardskel_obj = CVmObj();
    bboardskel_obj.setScale(room_scale * bboardskel_scale);
    bboardskel_obj.LinkToMesh(bboardskel_mesh);
    bboardskel_obj.setPosition(-1.5, 0.4, 1.5);
    bboardskel_obj.Rotate(90, 1, 0, 0);
    bboardskel_obj.Rotate(-90, 0, 0, 1);

    /*bboardex*/
    bboardex_mesh = CVmNewMesh("data/bboardex.aam");
    bboardex_obj = CVmObj();
    bboardex_obj.setScale(room_scale * bboardex_scale);
    bboardex_obj.LinkToMesh(bboardex_mesh);
    bboardex_obj.setPosition(1.5, 0.3, 1.5);
    bboardex_obj.Rotate(90, 0, 0, 1);

    room_mesh = CVmNewMesh("data/room3.aam");
    room_obj = CVmObj();
    room_obj.LinkToMesh(room_mesh);
    room_obj.setScale(room_scale);
    // 0
    room_obj.setPosition(0, -1, 1.5);

    /* load the skeleton parts */
    for(var i=0; i<len(skel_part_names); i++) {
        var mesh = CVmNewMesh("data/" + skel_part_names[i]);
        var obj = CVmObj();
        obj.LinkToMesh(mesh);

        obj.setScale(skel_scale);
    // to do bring skeleton mprosta sto z axona
        obj.setPosition(skel_part_pos[i] * skel_scale - skel_offs * room_scale);

        aadd(skel_parts, obj);
    }

    glEnable(GL_NORMALIZE);
}

function OnFrame()
{

    SetClearColor(0,0,0);

    SceneBegin();

    DrawGrid([0.5, 0.5, 0.5], 100);


    room_obj.Draw();
    bboard_obj.Draw();
    bboardskel_obj.Draw();
    bboardex_obj.Draw();
    ptr_obj.Draw(VR_CALC_MATRIX);

    if(show_bbox) {
        ptr_obj.DrawBoundingBox();
    }

    for(var i=0; i<len(skel_parts); i++) {
        skel_parts[i].Draw(VR_CALC_MATRIX);
        if(hl_obj == i) {
            if(correct(i)) {
                skel_parts[i].ModulateMaterials(0.3, 1.0, 0.5);
            } else {
                skel_parts[i].ModulateMaterials(1.0, 0.5, 0.3);
            }
        } else {
            skel_parts[i].ModulateMaterials(1.0, 1.0, 1.0);
        }
        if(show_bbox) {
            skel_parts[i].DrawBoundingBox(VR_NOSHADING);
        }
    }

    SceneEnd();
}

//Moving the Control
function MoveControl()
{
    var bn_press = false;

    if(in_cave) {
        ptr_pos = getTrackerPos(1);

        if(getTrackerButton(1) == 1) {
            bn_press = true;
        }
        if(getTrackerButton(1) == 8) {
            scatter();
        }
    }
    else {
        var move_dist = 0.02;

        var pressedKey = Keyboard();
        switch(pressedKey){
        case "a":
            ptr_pos.x -= move_dist;
            break;

        case "d":
            ptr_pos.x += move_dist;
            break;

        case "w":
            ptr_pos.y += move_dist;
            break;

        case "s":
            ptr_pos.y -= move_dist;
            break;

        case "e":
            ptr_pos.z -= move_dist;
            break;

        case "c":
            ptr_pos.z += move_dist;
            break;

        case "b":
            show_bbox = !show_bbox;
            break;

        case "-":
            skel_scale *= 0.9;
            if(skel_scale < 0.0) {
                skel_scale = 0.0;
            }
            update_skeleton();
            outputln("skeleton scale: ", skel_scale);
            break;

        case "=":
            skel_scale *= 1.1;
            update_skeleton();
            outputln("skeleton scale: ", skel_scale);
            break;

        case "[":
            room_scale *= 0.9;
            if(room_scale < 0.0) {
                room_scale = 0.0;
            }
            room_obj.setScale(room_scale);
            room_obj.setPosition(0, -room_scale, 0);
            update_skeleton();
            outputln("room scale: ", room_scale);
            break;

        case "]":
            room_scale *= 1.1;
            room_obj.setScale(room_scale);
            room_obj.setPosition(0, -room_scale, 0);
            update_skeleton();
            outputln("room scale: ", room_scale);
            break;

        case ",":
            snap_thres *= 0.9;
            outputln("snap threshold: ", snap_thres);
            break;

        case ".":
            snap_thres *= 1.1;
            outputln("snap threshold: ", snap_thres);
            break;

        case " ":
            bn_press = true;
            break;

        case "\b":
        case "\n":
        case "\r":
            scatter();
            break;
        }
    }

    if(bn_press) {
        if(picked_obj == -1) {
            picked_obj = find_collision();
        } else {
            if(correct(picked_obj)) {
                skel_parts[picked_obj].setPosition(target_pos(picked_obj));
            }
            picked_obj = -1;
        }
    }

    ptr_obj.setPosition(ptr_pos);

    if(picked_obj != -1) {
        skel_parts[picked_obj].setPosition(ptr_pos);
    } else {
        hl_obj = find_collision();
    }
}


function DownloadReady(RequestID)
{

}

function OnTimer()
{
    /* manage camera */
    if(!in_cave) {
        CameraMoveMouse();
    }
    MoveControl();

}

function OnExit()
{
}

function find_collision()
{
    for(var i=1; i<len(skel_parts); i++) {
        if(skel_parts[i].IsCollidingBBox(ptr_obj)) {
            return i;
        }
    }
    return -1;
}

function target_pos(idx)
{
    return skel_part_pos[idx] * skel_scale - skel_offs * room_scale;
}

function correct(idx)
{
    var pos = skel_parts[idx].getPosition();
    var targ = target_pos(idx);
    var dif = targ - pos;

    if(dif.x * dif.x + dif.y * dif.y + dif.z * dif.z < snap_thres * snap_thres) {
        return true;
    }
    return false;
}

function update_skeleton()
{
    for(var i=0; i<len(skel_parts); i++) {
        skel_parts[i].setScale(skel_scale);
        skel_parts[i].setPosition(skel_part_pos[i] * skel_scale - skel_offs * room_scale);
    }
}


function randvec()
{
    return [tofloat(rand(8192) - 4096) / 4096.0, tofloat(rand(8192) - 4096) / 4096.0, tofloat(rand(8192) - 4096) / 4096.0];
}

function scatter()
{
    for(var i=1; i<len(skel_parts); i++) {
        skel_parts[i].translate(randvec() * [room_scale * 1.0, room_scale * 0.9, room_scale * 0.8] * 0.5);
    }
}


// Camera manager (using mouse)
function CameraMoveMouse()
{
    static var InMouseR = false, InMouseL = false;
    static var PrecX = 0, PrecY = 0;
    // Change these values to modify the mouse sensitivity
    var TR_SENSITIVITY  = 0.0001 ;//* SceneScale;
    var ROT_SENSITIVITY = 0.01;
    // Mouse manager
    if(Mouse.ButtonL && !Mouse.ButtonR)
    {
        //====  Left Button: Camera rotation  ====//
        if (InMouseL)
        {
            CameraRotate(( Mouse.X-PrecX)*ROT_SENSITIVITY,0,1,0);
            CameraRotateABS((Mouse.Y-PrecY)*ROT_SENSITIVITY,1,0,0);
        }
        else
        {
            PrecX = Mouse.X;
            PrecY = Mouse.Y;
        }
        InMouseL = true;
        InMouseR = false;
    }
    else
    if(Mouse.ButtonR)
    {

        //====  Right Button: Camera translation  ====//
        if (InMouseR)
        {
            var CameraMatrix = CameraGetMatrix();
            var CameraPos = CameraGetPosition();
            if (!Mouse.ButtonL)
            //====  Translation on X and Z axis ====//
                CameraPos += CameraGetZAxis() * (Mouse.y-PrecY)*TR_SENSITIVITY + CameraGetXAxis() * (Mouse.X- PrecX)*TR_SENSITIVITY;
            else
            //====  Right + Left Button: Translation on Y axis ====//
                CameraPos -= CameraGetYAxis() * (Mouse.y-PrecY)*TR_SENSITIVITY;

            CameraSetPosition(CameraPos);
        }
        else
        {
            PrecX = Mouse.X;
            PrecY = Mouse.Y;
        }
        InMouseR = true;
        InMouseL = false;
    }
    else
    {
        InMouseR = false;
        InMouseL = false;
    }
    /*
    var CameraPos = CameraGetPosition();
    var CameraDir = CameraGetDirection();*/
}

function DrawGrid(col, size)
{
    /* let's not mess up current OpenGL status */
    glPushAttrib(GL_LIGHTING_BIT | GL_LINE_BIT | GL_CURRENT_BIT);

    glLineWidth(1);
    glDisable(GL_LIGHTING);
    glColor(col);
    var max = size / 2.0;
    var min = -max;
    var step = size / 10.0;

    glBegin(GL_LINES);
    for (var i = min; i <= max; i += step) {
        glVertex(i, 0, max);
        glVertex(i, 0, min);

        glVertex(max, 0, i);
        glVertex(min, 0, i);
    }
    glEnd();

    /* polite restoration of previous OpenGL status */
    glPopAttrib();
}

function onError()
{
}
function onEvent()
{
}


The skeleton model was downloaded from here.