There are some differences between the books and the films, so to get the full experience, I would read the books.
You would be able to understand most of the storyline if you jumped to the final books, but there are intricate plot points and scenes of character development that are missing from the films. Someone mentioned the Twins' jokes. The relationship between Bill and Fleur. The emotions Harry goes through. The real situation when Harry leaves Privet Drive for good. These are things the filmmakers just can't pack into the films due to the limited running time. In addition, there are some jokes that refer to other books, so if you want to be able to understand everything, read the books!
If you're worried about the length of the books, yes, they are long, but they're very easy reading. Rowling is a very descriptive writer, and so the pages fly by. She's able to make 800 pages seem like nothing because she adds so much detail. Sit back and enjoy it rather than skipping ahead!
Harry Potter fan, English student with prior courses in comparing literature and film