knee not towards right direction when the leg moves.

My inital thought is a weighting issue, but it only happens with IK right? I sent a request to access the blend file. 

I've put the link at the end, doesn't it work?

Yes,the problem is in Ik mode but I came up with a solution, I put a "Damped Track" on sheen bone and selected the knee pole bone as target and it fixed the problem,

but now there's another problem which is a slight transformation between "Pose Position" and "Rest Position" after I added the "Damped Track" on sheen bone,

this is a problem that happens with other constraints as well, not always but sometimes, 

one thing I do to fix it is to change the space( world, local, ...) on bone's constraints, but it can cause other issues sometimes, plus , not all constraints have the "Space" option in them,

Can you see where is the issue?

You have a couple of problems. 

First, lets start in edit mode.

1. Fix the roll. Select the thigh and shin bones. Press shift+n->Global X axis. 

2. Remove Z lead. Select the thigh bone. On N-panel under head put mouse cursor over X input field and press ctrl+C. Then move mouse cursor over X input field under Tail and press Ctrl+V. 

3. Select tail(Small end of the bone) of the sheen bone. Press shift+s->Cursor to selected. Then select body of the IK CTRL bone and press shift+s->Selection to cursor.

4. [Optional but a good practices] Line up knee pole with world. With mouse cursor over Z under head press ctrl+c. Then move mouse cursor to Z under Tail and press ctrl+V. Set Roll to 0.

Second, In pose mode.

1. Select Shin. On the IK Constraint change target bone from  Foot to the IK CTRL. 

Third, Additional information. What is happing is that there is to much of a bend along the joint's Z axis thus Blender moves in both directions(Z and X). The Joint is where the thigh and Shin meet. The joints Z axis moves along the global X axis which is why I copied and pasted the X axis of the head and tail.  You could also fix this by moving the joint further forward on the X axis, but this lead to having to reweight around the knee and calf. That's why I choose to do the Joint's Z axis. Fixing the roll just gives a better deform with deeper bend of the leg. As for the Pole target, it's just common practice to have it straight. I don't know if it really makes a difference in Blender. The reason I snapped the IK to the Shin is so that when you set the correct Target on the constraint the shin wouldn't jump and cause weird deformation in the ankle. 

You're right, The straighter the bones and more aligned to world space, the easier they are to rig and the less likely it is to make a difference between "Pose Position" and "Rest Position",

But in most of the characters, especially those in more realistic styles, the bones are oriented in different directions, and they don't all lay in a straight line, right?

It's not straight in relation to world origin. It's straight in relation of the chain of bones. A good example is the way you had it setup would be perfect for a bow legged person, because their knees do point outward. I choose to align along the world axis instead of aligning the shin to the thigh's position, because it was easier to describe in text. Bone Roll and the alignment of the bone's axis are very important in getting the expected movement while making it intuitive for animators. These become even more important when you add IK, because Blender relies on this to calculate how the bones should move/bend. As for Realistic, the best way is to break it down to parts and keep testing over and over as you go. I like to start at the hips and go thru the leg first. Then the basic foot. Then the body. Then the arm. Then the basic hand. Then go back to the foot and add advance features like foot roll. Then advance features of the hand. Then the neck and head. Then lastly the eyes. I've just started learning facial rigging so I'm not as adapted on that, but I can still trouble shoot problem in this area.