Yes, they will stay with you forever. I haven't tried this but I think they can still die if you don't feed them. If you truly don't want them anymore try putting them to bed without feeding them. It might work.

Also make sure their energy is under 20%

You starter horses can never be sold and they will have Philosopher's Stone so cannot die. This change was introduced a few years ago to stop people from ending up with no horses on their accounts. We were told it affects your gameplay and can get your game stuck as unplayable if you have no horses on your account.

You can just ignore them or groom, feed and bed them each day to collect droppings (or equus if you use a Ploutos Parchment) and for chance to collect APs each day. You could also use them as low skilled filler horse to help get wins for blupping.

Thank you both!

Oh well I guess it is what it is :P

They won't die because of the philosophers stone

Thank you