As long as you aren't reving over the redline by downshifting, there is no harm to the engine or gearbox. Geez, the're designed for that! The stress on the components is no different than accellerating - it's just reversed.
When accellerating, more gas/air is forced into the cylinder, which causes is to explode more violently, pushing faster on the camshaft, which speeds up the gearbox and the rear tire. There is a loss of energy in each of these steps - the pistons have friction within the cylinder walls and in the connection to the camshaft, the gearbox has internal resistance, alot of energy is lost through the flex of the chain. You don't notice this, though, since you are accellerating.
When using engine braking, the rear tire wants more speed from the gearbox, which transfers this to the camshaft, making it turn faster and causing the pistons to rise and fall faster, pulling more mix into the cylinders. Since each of these components introduces some resistance (none of these can transfer 100% of the energy onto the next component), the net result is an engine brake.