Just keep the icon pressed for a few second and they will start wiggling, and an "X" will also appear in the upper left corner of the icon. Press the "X" and it will ask if your sure you want to delete the app. Select "OK" and it will be deleted. If you change your mind you can always download the app again, free of charge.