Played all of Innocent Sin and currently playing Eternal Punishment, all this after already playing Persona 4 and then 3 by the way, so it's been a while but...
P2IS has easily the best main cast of ANY Persona game to the date, it is a small 5 man, 6 man group which you get to know right from the get go, so you have ample chances to screw around with them, fight, get new Persona and see them react and develop all throughout the story. They are all SO full of personality it's ridiculous, it is often said as a joke you would need to combine Yukari with Chie, Rise and Ann just to equate to Lisa Silverman (They all are different characters of course with different points to them, but Lisa is something else in terms or quirks and character).
The stakes to the setting to the tone and music are great, so far, the Persona 2 duology is outright the best one, no contest. Although, it is an old game and also experimental, it is rather archaic and not as easy to get as the Neo-Persona games, and it can be a hassle to some. It wasn't to me but it's a thing that happens.
I personally love it, even more after experiencing mainline SMT.
When it comes to Persona 1... I only did play the intro right after coming from P4 and P3, and could not fully get into it, but I plan to try it once more soon. After going through SMT1, 2, If, 3 and then 4, I believe it should be a far less complicated experience. I heard good things about it, but as the others have said, it's complicated and I would say even intimidating.