The "Photo" Problem

Everyone goes on and on about photo syncing services and what have you, but the one thing that unnerves me the most about iCloud and well, anything, is that I have to manage two libraries of photos.

I don't really care about the photo caps, and iCloud is generally faster/more stable than dropbox for this solution.

But when I delete a photo from camera roll I have to also delete it from photo stream, and vice versa. Am I just doing it wrong? I guess I can see why, in some very rare instance, you wouldn't want to delete from both. But for me I just wish I could delete from photo stream and never see it again.