Guilt, when it is our experience, is our experience until another one supersedes it. Our heads may understand the principle of vicarious sacrifice and the mechanism of forgiveness finished in the past, but our hearts feel guilt and remorse in the present and do so until they are alleviated. A battle rages between our minds and our hearts on the subject, because there is a difference between knowing something intellectually and feeling it emotionally. Believers in Christ can end up feeling shameful before God, even ostracized from him, despite knowing about the cross.
When Christians focus on their feelings of guilt they become prey for the enemy, who seeks to drive a wedge between them and God, ultimately to undermine their faith in God's forgiveness. Believers under the weight of guilt are on the edge of doubting their status before God. On the other hand, believers who are presumptuous, and ride roughshod over conscience, can undermine their own faith, sin against the grace of God, and wind up in the same place anyhow. God help us!
Guilt is not a necessarily bad thing for a Christian, but I think it takes an experiential revelation of forgiveness for it not to be so in the long run. That doesn't mean we're not forgiven if we don't feel forgiven, or even that we're not forgiven until we confess our sin and ask. Forgiveness was attained on the cross and sealed by the resurrection. It's a fait accompli. The one who believes in Christ and what he accomplished through his passion is forgiven regardless of confession or feeling.
But given the timing of guilt and forgiveness, there will be occasions when the believer will have to hold on to the facts of forgiveness with an iron grip until the feelings of guilt crumble and give way to those facts. Sometimes guilt must hang on the cross of Christ until it dies.