I got the same symptoms in a 2004 Sienna.
Found someone in another forum posting that the Door Lock Release Motor is the problem;
PN: 8562008061 and he paid dealer $103.09 for it.
Searching at this website =>
I Need Toyota Parts - then we got em
Found this.
Item Number MSRP Core Price Price
8562008061 $103.09 $0.00 $75.62
MOTOR ASSY, SLIDE DO
Now I just need some advice, pictures or pdf of procedure on how to replace. Affected door is the passenger side sliding door.
How much should this work cost at the dealer?
Thanks in advance!