In this post I’m going to talk about anchoring the terrain to a surface plane and getting the terrain oriented to match the surface plane. I’m not resizing the terrain to match the given plane just yet because I discovered a couple of odd bugs with my terrain regarding this. I’ll save that stuff for the next post.
Selecting a Surface
The first step in anchoring the terrain to a surface plane is to pick a surface. I’ve already gone over how to capture user interaction so I’ll just go over a bit of what I’ve done to select a specific plane the user is looking at when they tap.
bool SurfacePlaneRenderer::CaptureInteraction(SpatialInteraction^ interaction) {
if (m_planeList.size() < 1) {
// No planes to intersect with.
return false;
}
// Get the user's gaze
auto gaze = interaction->SourceState->TryGetPointerPose(m_coordinateSystem);
auto head = gaze->Head;
auto position = head->Position;
auto look = head->ForwardDirection;
// For each surface plane quad, check if the gaze intersects it.
// track the closest such intersection.
int index = -1; // index of closest intersected plane.
float dist = D3D11_FLOAT32_MAX; // initial distance value.
int i = 0;
for (auto p : m_planeList) {
// the BoundingOrientedBox object has built in intersection tests we can use to test it against our
// gaze.
float d = 0;
XMFLOAT3 _pos = XMFLOAT3(position.x, position.y, position.z);
XMFLOAT3 _dir = XMFLOAT3(look.x, look.y, look.z);
XMVECTOR pos = XMLoadFloat3(&_pos);
XMVECTOR dir = XMLoadFloat3(&_dir);
p.bounds.Intersects(pos, dir, d);
if (d > 0 && d < dist) {
dist = d;
index = i;
}
++i;
}
// if we have an index > -1, then we need to handle this interaction.
if (index > -1) {
// if so, handle the interaction and return true.
m_intersectedPlane = index;
m_gestureRecognizer->CaptureInteraction(interaction);
return true;
}
// if it doesn't intersect, then don't handle the interaction and return false.
return false;
}
We loop through all of the planes in our list and test our gaze ray against the BoundingOrientedBox of the plane. This is quite handy as it appears to handle all of the transformations to make sure the box and gaze are in the same space.
Because multiple planes could lay along the path of the ray, we also need to find the intersection point the shortest distance along the ray, returned by the intersection test.
Once we know which plane was selected, we can’t actually do anything with that data yet because we’re inside the SurfacePlaneRenderer class and we don’t actually know about the terrain here. This function returns us to the Main class’ OnInteractionDetected() method.
void HoloLensTerrainGenDemoMain::OnInteractionDetected(SpatialInteractionManager^ sender, SpatialInteractionDetectedEventArgs^ args) {
// if the terrain does not exist, check whether the interaction was with a surface plane.
// if the terrain does exist, check whether the interaction was with the terrain.
// in either case, if the interaction was not with the object, check it with the guiManager.
if (!m_terrain) {
if (!m_planeRenderer->CaptureInteraction(args->Interaction)) {
m_guiManager->CaptureInteraction(args->Interaction);
}
} else {
if (!m_terrain->CaptureInteraction(args->Interaction)) {
m_guiManager->CaptureInteraction(args->Interaction);
}
}
}
We can’t initialize the terrain here, either, because we don’t yet know whether the interaction we captured was actually a tap. We have to wait for the SurfacePlaneRenderer’s OnTap event to be triggered. When it is, we literally just set a variable called wasTapped so we can check on the next frame update whether the user has selected a surface. On the next call to Main’s Update() method, we do the following:
// if we haven't generated a terrain yet, check if we have recently
// tapped on a surface plane;
if (!m_terrain && m_planeRenderer->WasTappedRecently()) {
auto anchor = m_planeRenderer->GetAnchor();
auto dimensions = m_planeRenderer->GetDimensions();
auto orientation = m_planeRenderer->GetOrientation();
m_terrain = std::make_unique<Terrain>(m_deviceResources, 0.6f, 0.6f, 4, anchor, orientation);
}
As you can see, we grab an anchor representing the center of the surface plane, the orientation of the plane, and the dimensions of the plane. As I mentioned at the start of this post, I’m not dealing with the dimensions yet because of bugs I need to fix. But we can look at the other two.
Anchoring the Terrain
Our call to GetAnchor() is simple enough. It simply creates a new SpatialAnchor within the initial coordinate system we used to generate the planes, centered on the center of the plane.
SpatialAnchor^ SurfacePlaneRenderer::GetAnchor() {
auto plane = m_planeList[m_intersectedPlane];
auto center = plane.bounds.Center;
// create the anchor at the plane's center, relative to the coordinate system extant at the time of the
// plane's creation.
return SpatialAnchor::TryCreateRelativeTo(m_coordinateSystem, float3(center.x, center.y, center.z));
}
I discovered back in Part 7 that I needed these SpatialAnchors so we’ve already talked about how they work and what the Terrain code looks like with regards to using them. There’s not much to it anyway. Each frame we update the Terrain with the current coordinate system and generate a transformation matrix from the anchor’s coordinates to the current coordinates.
We can set the terrain’s location relative to the anchor pretty much wherever we want, so I’ve simply centered it based on the height and width of the terrain.
SetPosition(float3(-w / 2.0f, -h / 2.0f, 0.0f));
This will locate the terrain directly in the middle of the plane. This has the noticeable affect on the surfaces I’ve been using of putting the terrain inside the surface.
Note how the terrain is rising up out of the bed and you can’t see the edges.
This isn’t great looking to me, but it isn’t really avoidable unless I change the snapToGravityThreshold of the Plane Finding algorithm. I’ll play with this at some point, but the wrong value will cause the bed and bench I’m using to test to not be found as planes. If I use the floor or another smooth surface, the plane will rest correctly on top. Maybe I’ll add a user-adjustable offset value to the future GUI to deal with this.
Orienting the Terrain
I’m fairly certain this is working, but I won’t be 100% until I have sizing the terrain working. Right now, because the terrain is perfectly square, it’s kind of hard to tell if the terrain is oriented along the correct axes. It is definitely lined up, as you’ll see, it just might be perpendicular rather than parallel.
First things first, we need to get the orientation of the surface plane and pass it to the Terrain.
XMFLOAT4X4 SurfacePlaneRenderer::GetOrientation() {
auto plane = m_planeList[m_intersectedPlane];
XMMATRIX world = XMMatrixRotationQuaternion(XMLoadFloat4(&plane.bounds.Orientation));
XMFLOAT4X4 transform;
XMStoreFloat4x4(&transform, world);
return transform;
}
We then want to use this orientation to rotate our terrain to match the surface. The simplest way to do this is to just add this to the modelToWorld matrix we’re passing to the shaders.
// Position the terrain.
// Transform to the correct coordinate system from our anchor's coordinate system.
auto tryTransform = m_anchor->CoordinateSystem->TryGetTransformTo(coordinateSystem);
XMMATRIX transform;
if (tryTransform) {
// If the transform can be acquired, this spatial mesh is valid right now and
// we have the information we need to draw it this frame.
transform = XMLoadFloat4x4(&tryTransform->Value);
} else {
// just use the identity matrix if we can't load the transform for some reason.
transform = XMMatrixIdentity();
}
// add in the orientation of the terrain.
XMMATRIX orientation = XMLoadFloat4x4(&m_orientation);
transform = orientation * transform;
// Get the translation matrix.
const XMMATRIX modelTranslation = XMMatrixTranslationFromVector(XMLoadFloat3(&m_position));
// The view and projection matrices are provided by the system; they are associated
// with holographic cameras, and updated on a per-camera basis.
// Here, we provide the model transform for the sample hologram. The model transform
// matrix is transposed to prepare it for the shader.
XMStoreFloat4x4(&m_modelConstantBufferData.modelToWorld, XMMatrixTranspose(modelTranslation * transform));
The result was pretty surprising, at first.
ummm….?
It took me a few minutes to remember that the bounding box we’re taking this orientation from is defined as being initially oriented vertically. It needs to be rotated from that vertical position into the horizontal. Since our terrain is defined as already being horizontal, we wind up rotating it into the vertical.
Luckily, that’s a really simple fix. We can just define the terrain’s vertices to be initially vertical so the transform works correctly. Essentially, the terrain shares the same object space as the surface, so the same transforms can be applied. I won’t bother showing the code changes because all you need to do is switch from defining your vertices as (x, 0, z) to (x, y, 0). And, of course, remember to update the vertex shaders to set the correct member with the height value.
Well, I suppose that’s closer.
Ok… So now we’re upside down? That’s a little bit odd. I didn’t bother too hard trying to wrap my brain around how these surfaces are being defined that my terrain would wind up upside-down. Flipping the rotation matrix around the z-axis so that the terrain is upside-right is pretty simple. We just need to negate the correct row of the matrix.
// invert the z-axis of the orientation matrix because for some reason it is backwards to what we need. m_orientation._31 *= -1; m_orientation._32 *= -1; m_orientation._33 *= -1;
The terrain is anchored to the bench this time.
It’s a little hard to tell for certain if the terrain is oriented correctly here because I’ve turned the surface plane rendering off when the terrain is placed. I’ve turned it back on until I know I’ve got this working.
Definitely looks lined up now.
I still need to address the Terrain’s CaptureInteraction() method to deal with the change in orientation. I’ve updated the flip in axis, but not the whole rotation. The intersection code still works surprisingly well, but some quick testing proves that things aren’t actually lined up.
So that’s it for today. Next post, I’ll tackle sizing the terrain, which will probably just mean I’ll be going over the bugs I found in my code, since sizing the terrain should have just meant passing the new height and width of the terrain in.
For the latest code, see GitHub.
Traagen