Replacing front speakers

I'm wanting to replace the front speakers on my AU wagon, can I just take the speaker grill off, or does the whole door trim have to come off?

I have always taken the whole door trim off.

Cheers that should help