You can construct a BezierCurve by adding two control points to the list of vertex coordinates for the two vertices (p[[1]] and p[[2]]) incident to an edge.
In the following function bC with three arguments ({t1,t2,t3}), the two additional points are obtained by taking two points on the line joining the two vertex coordinates, t1 p[[1]] + (1-t1) p[[2]] and t2 p[[1]] + (1-t2) p[[2]] where 0<= t1, t2 <=1, and adding to these points {0,t3} or {t3,0} to ensure that the resulting four points are not collinear. The default values for the three arguments are {t1,t2,t3}= {1/3,2/3,1/3}.
ClearAll[bC] bC[t1_:(1/3),t2_:(2/3), t3_:(1/3)]:= With[{p=#}, BezierCurve[{p[[1]], t1 p[[1]]+(1-t1)p[[2]]+If[p[[1,1]]==p[[2,1]],{t3,0},{0,t3}], t2 p[[1]]+(1-t2)p[[2]]-If[p[[1,1]]==p[[2,1]],{t3,0},{0,t3}], p[[2]]}]]&; LayeredGraphPlot[ {{"i" -> "m", 1}, {"i" -> "...", 1}, {"i" -> "n", 1}, {"m" -> "j", 2}, {"..." -> "j", 2}, {"n" -> "j", 2}}, DirectedEdges -> True, VertexLabeling -> True, PlotStyle -> {FontSize -> 14}, EdgeRenderingFunction -> (Switch[#3, 1, {Dashed,Arrow[bC[][#],.1]}, 2, {Arrow[#1, .1]}] &)]

EdgeRenderingFunctionwithArrow. All you need to do is use a function that makes curved lines instead ofArrow. $\endgroup$"EdgeLayout"option. $\endgroup$