On a lightly floured board, roll out pastry about 1/83 inch thick.
Fit into a deep 9 inch pie pan; crimp edge.
Bake in a preheated 400° oven for 10 minutes.
Let cool.
Meanwhile, break cauliflower into flowerets and cut into 1/2 inch thick pieces. (You should have 4 cups.)
Steam over boiling water until just crisp tender (about 4 minutes).
Drain, plunge into cold water to cool, then drain again.
Spread almonds in a shallow pan and toast in a 350° oven for about 8 minutes or until lightly browned.
Place cauliflower in bottom of pastry shell and sprinkle with toasted almonds.
In a blender or food processor, whirl eggs, milk, and mayonnaise until smooth.
Add 1 1/4 cups of the cheese, along with pepper and nutmeg, and whirl briefly to mix.
Pour over cauliflower and nuts in pastry shell.
Sprinkle with remaining 3/4 cup cheese.
Bake on bottom rack of a 350° oven for 30 to 35 minutes or until a knife inserted in center comes out clean.
Let stand on a wire rack for 10 minutes before serving.