In your Position Players section on your lineup page, you’ll now see a switch for benching players that are not in their MLB starting lineup:
Once you turn this switch on, it will be on until you turn it off. You can go forward in terms of dates on the lineup page and see how the switch works - it stays on or off until you change its state and then propagates that state until you change it again.
The way the switch works when enabled is: if a player is in your lineup as a hitter and is not explicitly in their MLB starting lineup for that day 5 minutes before the officially scheduled game time, they will be moved to your bench for just that day. If a player is at any position other than one of the hitting positions (SP, RP, Bench, IL, Minors), they will not be affected.
This is a change to a pretty important part of the site, so I will be closely monitoring this logic for the first week of the season to make sure it is working as expected.