Hi Piyush,
It is a really good question.
You can see section 58(4), no deduction for any expenditure or allowance will be allowed for earning casual incomes of the nature u have referred.
"Is SET-OFF an allowance?"
I dont think it is technically an allowance, so there should be no restriction on set off...
But dept always thinks one step ahead, just download the e-return facility from income tax website, and try filling up the cells... Will the business loss be set off against the income from casual sources?
Same anomaly in law for unabsorbed depreciation vs salary... same solution...
there could be a gap in law, but the intent is clear thru the return preparing software....
Plus the taxation is direct like aditya sir pointed out...