This approach punishes labor effort and savings, but the latter effect is the more pernicious. Savings finances capital formation and research and development, which are crucial for economic growth.
The usual objection to eliminating taxes on capital income is that it would largely benefit high-income households, who get much of their income from interest, dividends and capital gains.
The fairness objection is not convincing, however, because American taxation of capital income leads capital to countries with lower rates of taxation. The big losers are then the people who might have earned wages and salaries in the businesses using that capital. The impact of capital taxation on the distribution of wealth is thus ambiguous in theory, and it appears to be small in practice.
Paul Ryan may have a hard time convincing the country to repeal taxation of interest, dividends and capital gains, since this approach looks like an excuse to reward the rich. On the basis of standard economics, however, Ryan is on firm ground.