Rear brake caliper help

[shawndog360]

Im haveing trouble with 2 things one is on the brake caliper shown in the picture broke a little peice if i replace the black rubber peice and the screw will it still work???

[shawndog360]

second peice

[400ex28]

If you rotate the carrier around the it will raise up and the bolt will go though fine.

[shawndog360]

but then my chain will get loosend up or it will tighten?