I haven't taken the engine apart to this degree, but it just so happens that last week Ron @ Whoosh started selling an OEM cam bucket replacement. To get to it, you need to remove the pieces this thread is talking about.
I have no idea whether or not you can leave the spongy thing off, maybe. But here's the video of that installation:
[video=youtube;izFXhyhvCHU]https://www.youtube.com/watch?v=izFXhyhvCHU[/video]