Part of the problem is that the StoryInit code is wrong. In order for $dice.random() to work, $dice needs to be an array, like this:
<<set $dice = ["[6][$dice+=6]", "[5][$dice+=5]"]>>
<<set $randomDice = $dice.random()>>
However, in your code above, $dice was a actually string ("[6][$dice+=6]", because the rest after the comma would be ignored due to it not being within square brackets). And since that .random() wouldn't have worked, that was why $randomDice was set to undefined.
That said, you can't use that "corrected" code anyways, because if you did that you'd only get one value ever from $randomDice, since that value would only get set once. Doing it that way doesn't keep giving you different random values.
For what I think you're trying to get, which is N rolls of six sided dice, you'd want to use a widget by creating a new passage with "widget" and "nobr" tags, and put this in it:
<<widget "rolls">>
<<capture _rollval>>
<<for _rollno = 0; _rollno < $args[0]; _rollno++>>
<<set _rollval = random(1, 6)>>
<<link `"roll " + _rollval` $args[1]>><<set $dice = _rollval>><</link>><br>
<</for>>
<</capture>>
<</widget>>
Then in your passage you could just do:
<<rolls 3 "next">>
and that would display three random rolls the player could pick from, and clicking any of those links would take them to the "next" passage and add the value of that roll to the $dice variable. You could change that number and the string in quotes to change the number of rolls and the passage that they link to. (Note that because the numbers are random, it's possible that some or all of the rolls could randomly be the same value.)
This works propertly because the <<capture>> macro "captures" the value of _rollval at the time the link is created. Without "capturing" it, _rollval would only have the last value it was set to, so clicking any link would act as though the last link was the one clicked every time.
Also, the "backticks" (the ` on the ~ key) within the <<link>> macro causes what what's inside those backticks to be evaluated. So, in the above code, the <<link>> macro sees `"roll " + _rollval` as a single parameter (e.g. something like "roll 5").
Hope that helps! :-)