Partial out all but i-th variable in variadic function in Clojure

I'm starting out learning Clojure, and was trying to implement some basic numerical derivative functions for practice. I'm trying to create a gradient function that accepts an n-variable function and the points at which to evaluate it. To do this in a "functional" style, I want to implement the gradient as a map of a 1-variable derivatives.

The 1-variable derivative function is simple:

(defn derivative 
"Numerical derivative of a univariate function."
[f x]
    (let [eps 10e-6] ; Fix epsilon, just for starters.
          ; Centered derivative is [f(x+e) - f(x-e)] / (2e)
          (/ (- (f (+ x eps)) (f (- x eps))) (* 2 eps))))

I'd like to design the gradient along these lines:

(defn gradient
"Numerical gradient of a multivariate function."
[f & x]
    (let [varity-index          (range (count x))
        univariate-in-i (fn [i] (_?_))] ; Creates a univariate fn
                                         ; of x_i (other x's fixed)
        ;; For each i = 0, ... n-1:
        ;; (1) Get univariate function of x_i
        ;; (2) Take derivative of that function
        ;; Gradient is sequence of those univariate derivatives.

        (map derivative (map univariate-in-i varity-index) x)))

So, gradient has variable arity (can accept any # of x's), and the order of the x's counts. The function univariate-in-i takes an index i = 0, 1, ... n-1 and returns a 1-variable function by partial-ing out all the variables except x_i. E.g., you'd get:

#(f x_0 x_1 ... x_i-1 % x_i+1 ... x_n)
                      ^
                     (x_i still variable)

map-ping this function over varity-index gets you a sequence of 1-variable functions in each x_i, and then map-ping derivative over these gets you a sequence of derivatives in each x_i which is the gradient we want.

My questions is: I'm not sure what a good way to implement univariate-in-i is. I essentially need to fill in values for x's in f except at some particular spot (i.e., place the % above), but programmatically.

I'm more interested in technique than solution (i.e., I know how to compute gradients, I'm trying to learn functional programming and idiomatic Clojure). Therefore, I'd like to stay true to the strategy of treating the gradient as a map of 1-d derivatives over partialed-out functions. But if there's a better "functional" approach to this, please let me know. I'd rather not resort to macros if possible.

Thanks in advance!

Update:

Using Ankur's answer below, the gradient function I get is:

(defn gradient
"Numerical gradient of a multivariate function."
[f & x]
    (let [varity-index     (range (count x))
          x-vec            (vec x)
          univariate-in-i 
             (fn [i] #(->> (assoc x-vec i %) (apply f)))]

       (map derivative (map univariate-in-i varity-index) x)))

which does exactly what I'd hoped, and seems very concise and functional.